Feng Yu

Posted on Sep 20

Ask PyData: A Source-Linked Agent for Python Data Library Decisions

#sanitychallenge #python #ai #mcp

Sanity Challenge Path One Submission

What I Built

Ask PyData is a Sanity-backed agent that answers library-selection and migration questions for the Python data stack — pandas, Polars, DuckDB. It only works well because the content is structured: every claim carries a sourceUrl, every version-sensitive answer is checked against versionNote documents first, and contradictory claims are surfaced as disputed instead of silently picked.

Why this domain: pandas 3.0 shipped in January 2026 (new default string dtype, Copy-on-Write, removed APIs). Polars 2.0 shipped September 2, 2026 (streaming engine now default). Generic web search serves stale blogs written against old versions. A structured, version-aware knowledge base answers "what changed and how do I migrate" deterministically, with sources.

Key capabilities

Version-aware answers — queries versionNote before answering anything version-specific.
Source-linked claims — every answer point cites its sourceUrl.
Contradiction surfacing — "Polars is 5x faster" is tagged disputed, not assumed true.
Structured comparisons — API equivalents and benchmarks are typed fields, so answers are reproducible, not fuzzy.

Demo

Three questions, answered by the agent over the hosted Sanity MCP:

Q1. What changed in pandas 3.0 and Polars 2.0?

The agent queries versionNote and returns:

pandas 3.0.0 — breaking: many APIs deprecated in 2.x were removed. source
pandas 3.0.0 — behavior-change: Copy-on-Write is now default; chained assignment semantics changed. source
pandas 3.0.0 — behavior-change: String dtype is now default (no longer NumPy object dtype). source
Polars 2.0.0 — new: full 1.x→2.0 migration guide. source
Polars 2.0.0 — behavior-change: streaming engine default for all LazyFrame queries; ~5x faster aggregate. source

Q2. How do I migrate pandas groupby/merge/fillna to Polars?

The agent queries apiEquivalent:

pandas	Polars	Note
`df.groupby(col).agg('x')`	`df.group_by(col).agg(pl.col('x'))`	aggregations must be wrapped in `pl.col()`
`df.fillna(value)`	`df.fill_null(value)`	Polars distinguishes null vs NaN
`pd.merge(a, b, on=k)`	`a.join(b, on=k)`	join API differs
`df.apply(func)`	`df.select([pl.col(c).map(func)])`	prefer vectorized expressions
`pd.read_csv(path)`	`pl.read_csv(path)`	lazy equivalent is `pl.scan_csv()`

source: Polars pandas migration guide

Q3. Is "Polars is 5x faster" trustworthy?

The agent queries comparisonClaim and returns: status = disputed, source = the Polars 2.0 announcement post. The agent does not repeat the marketing claim as fact — it flags it.

Full machine-readable transcript: agent/transcript.txt

Code

Repository: https://github.com/fengyuGbt/ask-pydata

ask-pydata/
├── sanity/
│   ├── sanity.config.ts
│   └── schemaTypes/
│       ├── library.ts
│       ├── versionNote.ts
│       ├── apiEquivalent.ts
│       ├── migrationGuide.ts
│       ├── performanceBenchmark.ts
│       └── comparisonClaim.ts
├── agent/
│   ├── ask_pydata.py      # Python CLI MCP client
│   └── transcript.txt     # recorded Q&A run
└── README.md

Run it:

cd agent
python -m venv venv && source venv/bin/activate
pip install mcp httpx

# Token read from ~/.copilot/mcp-config.json (set up by `npx sanity mcp configure`)
python ask_pydata.py versions
python ask_pydata.py migrate
python ask_pydata.py controversy

How I Used Sanity

Knowledge modeling — 6 document types

library — pandas / Polars / DuckDB, with currentVersion and engine (eager vs lazy).
versionNote — per-version changes, with changeType enum: breaking / deprecated / new / behavior-change.
apiEquivalent — fromApi → toApi with semanticDiff and sourceUrl.
migrationGuide — longer-form step-by-step migration narratives.
performanceBenchmark — numeric value + unit + environment, so speed claims are queryable.
comparisonClaim — status: confirmed | disputed | deprecated, so contested claims are not silently trusted.

MCP usage

Configured with npx sanity mcp configure against the hosted endpoint https://mcp.sanity.io.
The Python client calls query_documents with a resource block {projectId: "654gu2bk", dataset: "production"} and a GROQ query.
Every returned document carries sourceUrl; answers are built from those fields, never from general LLM knowledge.

Why structured content matters here

"What changed in pandas 3.0" is now a GROQ query, not a web search through stale 2023 blog posts.
Source linking preserves provenance — the answer is the evidence.
Editing one versionNote in Studio fixes every future answer; no prompt retraining needed.

Build Log (honest, warts and all)

Built in one evening on a remote WSL2 (Ubuntu 24.04) box, driven over SSH. Real friction included:

WSL had no Node. npm pointed at a Windows mount path. Fixed by sourcing the existing nvm install and using Node 22.
npx sanity init --template clean gave a Studio skeleton; I hand-wrote the six schema types in schemaTypes/.
sanity dataset import wants NDJSON, not a JSON array. First batch failed on a pretty-printed array; rewrote as newline-delimited objects.
The @sanity/agent-context Studio plugin is not compatible with Sanity 6.x. It pins @sanity/icons@^3 while Sanity 6 ships @sanity/icons@^5, so Vite build dies with 29 MISSING_EXPORT errors. Fell back to npx sanity mcp configure, which points at the hosted https://mcp.sanity.io endpoint — simpler anyway.
The hosted MCP is HTTP, not stdio. The Python client needed mcp.client.streamable_http.streamable_http_client with a custom httpx.AsyncClient carrying the Authorization: Bearer header.
The Sanity read token must never be committed. The Python script reads it from ~/.copilot/mcp-config.json or SANITY_MCP_TOKEN, so the repo stays clean.

Takeaway: structured modeling was the easy part; the friction was all in transport and version compatibility.

Sanity Project Details

Sanity Project ID: 654gu2bk
Dataset: production
MCP endpoint: https://mcp.sanity.io
Dataset preview: https://www.sanity.io/manage/project/654gu2bk/datasets/production

Schema snapshot — versionNote (the core of version-awareness):

export const versionNote = {
  name: 'versionNote',
  title: 'Version Note',
  type: 'document',
  fields: [
    { name: 'version', type: 'string' },
    { name: 'changeType', type: 'string',
      options: { list: ['breaking', 'deprecated', 'new', 'behavior-change'] } },
    { name: 'summary', type: 'string' },
    { name: 'sourceUrl', type: 'url' },
  ],
}

GROQ filter used by the agent:

*[_type in ["library","versionNote","apiEquivalent",
            "migrationGuide","performanceBenchmark","comparisonClaim"]]

Agent Session

The recorded transcript (public): https://github.com/fengyuGbt/ask-pydata/blob/master/agent/transcript.txt

It contains the three Q&A runs above — version notes, API equivalents, and the disputed benchmark claim — each with the exact sourceUrl returned by the MCP query.

Built for the Sanity Challenge on dev.to.

DEV Community