DEV Community

jamilxt
jamilxt

Posted on

Build a Web-Grounded AI Research Tool with DeepSeek Search and Fetch APIs

An AI answer can sound current while quietly relying on yesterday's knowledge. For a question like “What changed in this library this week?”, a model needs two separate abilities: discover relevant pages and read the pages it found. Until recently, teams commonly assembled those pieces from separate search and extraction services, or delegated them to a framework whose behavior was difficult to inspect.

On October 17, 2026, DeepSeek opened its in-house search engine through two direct APIs: Search returns structured web results, and Fetch retrieves one URL as Markdown. The listed price is $0.75 per 1,000 successful requests for each API. That is a fresh building block, not a complete research agent. This tutorial turns it into one small, inspectable Python command-line tool.

The article is based on the public API documentation and release note. It does not claim benchmark results or personal testing against competing search providers. Treat the code as an implementation pattern, then verify behavior and current pricing in the linked documentation before using it in production.

What we are building

The program will:

  • Search for a question and print a bounded set of results.
  • Let you choose which result pages to fetch, rather than reading every result automatically.
  • Keep each fetched page's title and URL beside its Markdown content.
  • Bound the response size before you pass anything to a model.
  • Fail clearly when the key is missing, the service returns an HTTP error, or the response shape is unexpected.

This separation matters. Discovery and retrieval are external calls with cost, latency, and policy limits. Keeping them explicit makes it possible to add approval, caching, logging, and source review without hiding those decisions inside a prompt.

The API surface, in two calls

The official guide documents POST https://api.deepseek.com/v1/tools/search and POST https://api.deepseek.com/v1/tools/fetch. Both use a bearer API key and JSON. Search accepts a required query, an optional result count from 1 to 50, and need_content; Fetch accepts a URL and returns a title plus Markdown content. Search's count controls result quantity, not the request price.

The billing detail is easy to misread: Search and Fetch are each priced at $0.75 per 1,000 successful requests. A successful Search is billable when it returns HTTP 200 with non-empty results. A Fetch is billable when it returns page content. Empty searches, failed requests, and timeouts are not billed, according to the guide. Fetching five pages after one search therefore uses six API calls, not one. The October 17 release note separately says server-side Search and Fetch calls made through DeepSeek Harness, Claude Code, or Codex are currently free, with billing scheduled to begin October 31, 2026 at 16:00 UTC. This tutorial calls the standalone endpoints directly, so do not assume that exemption applies to it.

The release note says the newly opened endpoint is powered by DeepSeek's in-house search engine. That describes the provider's announcement, not an independent evaluation of index coverage or result quality. We'll measure those ourselves for the queries that matter to an application.

Step 1: Prepare a Python environment

The example uses only Python's standard library, so no package installation is needed. Create a file named research.py and provide the API key through an environment variable. Do not paste a real key into source code or commit it.

export DEEPSEEK_API_KEY="your-key"
python3 research.py "What changed in Python's asyncio this month?"
Enter fullscreen mode Exit fullscreen mode

On Windows PowerShell, set the variable for the current shell with $env:DEEPSEEK_API_KEY="your-key". The environment variable is a convenient local example, not a full secrets-management strategy for a deployed service.

Step 2: Make a small authenticated JSON client

The HTTP helper below centralizes authentication, timeouts, JSON encoding, and error handling. A timeout is essential: external search should not be allowed to hang a worker indefinitely.

import json
import os
import sys
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen

BASE_URL = "https://api.deepseek.com/v1/tools"
API_KEY = os.environ.get("DEEPSEEK_API_KEY")


def post_json(path: str, payload: dict) -> dict:
    if not API_KEY:
        raise RuntimeError("Set DEEPSEEK_API_KEY before running this script")

    request = Request(
        f"{BASE_URL}/{path}",
        data=json.dumps(payload).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {API_KEY}",
            "Content-Type": "application/json",
        },
        method="POST",
    )

    try:
        with urlopen(request, timeout=20) as response:
            body = response.read(2_000_000)
    except HTTPError as exc:
        detail = exc.read(4_000).decode("utf-8", errors="replace")
        raise RuntimeError(f"DeepSeek returned HTTP {exc.code}: {detail}") from exc
    except URLError as exc:
        raise RuntimeError(f"Could not reach DeepSeek: {exc.reason}") from exc

    try:
        return json.loads(body)
    except json.JSONDecodeError as exc:
        raise RuntimeError("DeepSeek returned invalid JSON") from exc
Enter fullscreen mode Exit fullscreen mode

The response read is capped at two megabytes to avoid unbounded memory use. The cap is a local safety choice, not a DeepSeek API limit. The error body is also capped because diagnostic responses should not become another uncontrolled input.

Step 3: Search and inspect the result list

Add a search function that asks for a small set of results and validates the response before using it.

def search(query: str, count: int = 5) -> list[dict]:
    query = query.strip()
    if not query or len(query) > 128:
        raise ValueError("Query must contain 1 to 128 characters")
    if not 1 <= count <= 50:
        raise ValueError("count must be between 1 and 50")

    data = post_json("search", {
        "query": query,
        "count": count,
        "need_content": False,
    })
    results = data.get("results")
    if not isinstance(results, list):
        raise RuntimeError("Search response did not contain a results list")
    return results
Enter fullscreen mode Exit fullscreen mode

The API also documents search operators such as site:, -site:, quoted phrases, lang:, after:, and before:. For example, a query can include site:docs.python.org asyncio TaskGroup after:2026-01-01. Use operators to narrow the information need, but do not treat a filtered result set as proof that no other relevant page exists.

Print results with their index, title, and URL. Let a human decide which pages are worth retrieving:

def show_results(results: list[dict]) -> None:
    for index, item in enumerate(results):
        print(f"[{index}] {item.get('title', '(untitled)')}")
        print(f"    {item.get('url', '')}")
        print(f"    {item.get('snippet', '')}\n")
Enter fullscreen mode Exit fullscreen mode

A production agent could make this selection itself, but it should still expose the chosen URLs and why they were selected. Search result summaries are useful for ranking and triage; they are not a substitute for reading the cited source when accuracy matters.

Step 4: Fetch selected pages and keep provenance

Fetch one page at a time. The content value is Markdown, and the API documentation notes that availability depends on the target site's access policies. Some URLs will not produce usable content. Handle those failures as per-source failures rather than failing the entire research task.

def fetch(url: str) -> dict:
    if not url.startswith(("https://", "http://")):
        raise ValueError("Only HTTP and HTTPS URLs are accepted")

    data = post_json("fetch", {"url": url})
    title = data.get("title", "")
    content = data.get("content", "")
    if not isinstance(content, str) or not content.strip():
        raise RuntimeError(f"No readable Markdown returned for {url}")

    return {
        "title": title,
        "url": url,
        "content": content,
    }
Enter fullscreen mode Exit fullscreen mode

For this simple command-line version, URL validation is deliberately basic. An application that accepts URLs from untrusted users needs stronger defenses against server-side request forgery, redirects to private network addresses, and unsafe schemes. Do not expose a raw fetch endpoint to arbitrary clients without reviewing that threat model.

The crucial design choice is that each document carries its source URL. Do not concatenate pages into one anonymous blob. Once provenance is lost, a model can produce a plausible statement while you can no longer tell which source supports it.

Step 5: Assemble a bounded research packet

A search can return up to 50 results, and fetched pages can be long. Before sending content to a model, enforce limits both per page and across the complete packet.

def make_packet(documents: list[dict], per_page_chars: int = 8_000) -> str:
    sections = []
    for doc in documents:
        body = doc["content"][:per_page_chars]
        sections.append(
            f"## {doc['title']}\n"
            f"Source: {doc['url']}\n\n"
            f"{body}"
        )
    return "\n\n---\n\n".join(sections)
Enter fullscreen mode Exit fullscreen mode

A character limit is not a token limit. If your model has a strict context budget, tokenize the packet with that model's tokenizer and reserve space for instructions and the answer. Also consider removing navigation and boilerplate, but keep the original URL and enough source text to verify key claims.

A responsible research prompt can ask the model to answer only from the supplied documents, attach a source URL to each factual claim, and say when the sources do not answer the question. That prompt helps, but it does not guarantee truth. The application still needs to show the underlying evidence and allow verification.

Step 6: Wire it together

The entry point searches, displays candidates, then fetches the first two. To keep the example runnable in one command, it selects those automatically; in an interactive application, insert a review or relevance-selection step here.

def main() -> int:
    if len(sys.argv) < 2:
        print('Usage: python3 research.py "your question"', file=sys.stderr)
        return 2

    question = " ".join(sys.argv[1:])
    try:
        results = search(question, count=5)
        if not results:
            print("No search results.")
            return 0

        show_results(results)
        documents = []
        for item in results[:2]:
            url = item.get("url", "")
            if not url:
                continue
            try:
                documents.append(fetch(url))
            except (RuntimeError, ValueError) as exc:
                print(f"Skipping {url}: {exc}", file=sys.stderr)

        if not documents:
            print("No pages could be fetched.", file=sys.stderr)
            return 1

        print("\n=== Research packet ===\n")
        print(make_packet(documents))
        return 0
    except (RuntimeError, ValueError) as exc:
        print(str(exc), file=sys.stderr)
        return 1


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

In a real agent, avoid blindly fetching the first two results. Rank candidates using transparent rules, require approval for sensitive sources, and cache repeated URLs when policy allows. Record request IDs or timestamps if the provider returns them in future versions, and keep enough logs to explain where an answer came from.

Cost and operating limits

With the current listed rate, one successful Search plus two successful Fetch calls costs $0.00225. That is arithmetic from the published per-request prices, not a promise about the total cost of a complete AI answer. Model inference, retries, and any other services are separate. Repeated agent loops can turn tiny per-call fees into meaningful usage, so count actual requests and set account-level budget alerts where available.

The guide lists a 5-queries-per-second account limit for Search and a separate 5-queries-per-second limit for Fetch. Fetch also has a per-domain limit. Requests beyond those limits can receive HTTP 429. Add bounded retries with exponential backoff and jitter in production; do not retry indefinitely, and do not retry non-retryable errors such as an invalid key or malformed request.

The API docs also state that Fetch can fail because a URL is unavailable, blocked by the site's policy, unsupported, or has no extractable Markdown. A search result is not permission to ignore the target site's access rules. Keep the source site's terms and applicable policies in your review process.

A checklist before shipping

  • Keep credentials out of code and logs. Use a secret manager in deployed environments and redact authorization headers.
  • Set request timeouts and response-size limits. External pages and services are not controlled by your application.
  • Preserve provenance. Keep title, URL, and content together through retrieval, summarization, and display.
  • Bound cost. Count searches and fetches, cap results and retries, and watch actual usage.
  • Treat fetched text as untrusted input. A page can contain prompt-injection instructions. Retrieved text is data, never authority to change system policy or invoke tools.
  • Separate search from action. Reading a page should not automatically authorize an agent to run code, send messages, or change records.
  • Test failures. Cover empty results, HTTP 401, 429, fetch rejection, timeouts, malformed JSON, and truncated content.
  • Check primary sources. Search summaries can help find evidence, but verify consequential claims against the original page.

The useful takeaway

DeepSeek's Search and Fetch endpoints make a formerly multi-provider web-retrieval path available behind two small JSON calls. The interesting part is not just the low per-request price. It is that the boundary is visible: search finds candidates, fetch reads selected pages, and your code decides what enters a model's context.

That boundary gives you a practical place to add source review, provenance, prompt-injection defenses, caching, and budgets. Start with a narrow question and a few sources. Measure whether the results are actually useful for your workload before turning the endpoints into an automatic research loop.

I write about practical AI infrastructure and developer tools. Subscribe if you want more build-focused guides like this.

Have you built web search into an agent yourself? What is your rule for deciding which pages are trustworthy enough to enter its context?

Sources

Top comments (2)

Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

Search then fetch is the right split. The cost hides in the fetch: a raw page carries navigation and footers, and the model keeps them in the history for every later turn. Reduce each page to its article body before it enters the context. tokenade web <url> does that reduction, if you'd rather not write one.

Collapse
 
arcwrightai profile image
arcwrightai •

Building this with DeepSeek's APIs sounds solid—pairing search with fetch for grounded answers is the right call. If you're on Windows, Arcwright makes wiring those calls into a working app a lot faster.