How to Scrape Perplexity Answers and Citations

Perplexity is useful because its answers expose the source layer. For developers building GEO dashboards, content research tools, or brand-visibility monitors, the citation list can be more valuable than the prose answer itself.

The hard part is collecting that surface consistently. The consumer interface renders dynamically, the answer and sources arrive over time, and the data you want is spread across answer text, web results, related prompts, and rich media.

The Scrapeless Perplexity actor turns that surface into structured JSON. You send a prompt, country, and web-search setting to one HTTP endpoint; the response arrives inside the same {status, task_id, task_result} envelope used by the other LLM Chat Scraper actors.

What the Perplexity actor returns

The main fields are platform-specific even though the top-level envelope is shared.

Field	Purpose
task_result.prompt	The exact prompt associated with the capture
task_result.result_text	The generated answer with its formatting and citation markers
task_result.web_results[]	Ranked web sources with names, URLs, and snippets
task_result.related_prompt[]	Suggested follow-up questions
task_result.media_items[]	Images, videos, maps, and other rich media when present

Treat every optional array as nullable. A research prompt may return a rich web result set, while another query may produce no media at all.

The current field reference is available in the Perplexity Scraper documentation.

Prerequisites

You need:

• A Scrapeless account and API key
• curl for the first request
• Python 3.10 or newer for the complete example
• The Python requests package

Set the key in your shell instead of placing it in the script:

export SCRAPELESS_API_KEY=your_api_token_here

Make a Perplexity request with curl

Perplexity uses the scraper.perplexity actor. The request body has the shared {actor,input} shape.

curl -sS -X POST "https://api.scrapeless.com/api/v2/scraper/execute" \
  -H "Content-Type: application/json" \
  -H "x-api-token: ${SCRAPELESS_API_KEY}" \
  -d '{
    "actor": "scraper.perplexity",
    "input": {
      "prompt": "What are the best web scraping APIs for AI agents?",
      "country": "US",
      "web_search": true
    }
  }'

The three input fields serve different roles:

• prompt is the question to send.
• country pins the residential exit market.
• web_search controls whether the run uses Perplexity's web-grounded answer path.

If the data will be compared over time, keep all three values fixed within a series.

Understand the response

The following sample is illustrative. It shows the documented field layout without claiming that these values came from a specific live prompt.

{
  "status": "success",
  "task_id": "5ad71f2c-...",
  "task_result": {
    "prompt": "What are the best web scraping APIs for AI agents?",
    "result_text": "A useful API for an AI agent should return structured data... [1]",
    "related_prompt": [
      "How do AI agents use web scraping APIs?"
    ],
    "web_results": [
      {
        "name": "Example source",
        "url": "https://example.com/source",
        "snippet": "An example source description."
      }
    ],
    "media_items": []
  }
}

Store task_id with the payload. It gives each capture a stable audit key when the same prompt is run more than once.

Write a reusable Python client

This client returns a normalized record while preserving the raw task_result for later analysis.

import os
import time
from urllib.parse import urlparse

import requests

ENDPOINT = "https://api.scrapeless.com/api/v2/scraper/execute"


def normalize_domain(url: str) -> str:
    return urlparse(url).netloc.lower().removeprefix("www.")


def scrape_perplexity(
    prompt: str,
    country: str = "US",
    web_search: bool = True,
) -> dict:
    response = requests.post(
        ENDPOINT,
        headers={
            "Content-Type": "application/json",
            "x-api-token": os.environ["SCRAPELESS_API_KEY"],
        },
        json={
            "actor": "scraper.perplexity",
            "input": {
                "prompt": prompt,
                "country": country,
                "web_search": web_search,
            },
        },
        timeout=300,
    )
    response.raise_for_status()
    payload = response.json()
    result = payload.get("task_result") or {}

    citations = []
    for position, source in enumerate(result.get("web_results") or [], start=1):
        url = source.get("url") or ""
        citations.append(
            {
                "position": position,
                "title": source.get("name") or "",
                "url": url,
                "domain": normalize_domain(url),
                "snippet": source.get("snippet") or "",
            }
        )

    return {
        "platform": "perplexity",
        "prompt": prompt,
        "country": country,
        "web_search": web_search,
        "captured_at": int(time.time()),
        "status": payload.get("status"),
        "task_id": payload.get("task_id"),
        "answer": result.get("result_text") or "",
        "citations": citations,
        "related_prompts": result.get("related_prompt") or [],
        "media_items": result.get("media_items") or [],
        "raw": result,
    }

The adapter assigns a citation position based on the returned order. That makes it easy to compare source prominence without parsing citation markers from the answer text.

Print an answer and its sources

Add a small command-line entry point:

if __name__ == "__main__":
    capture = scrape_perplexity(
        "What are the best web scraping APIs for AI agents?"
    )

    print(capture["answer"])
    print("\nSources")

    for source in capture["citations"]:
        print(
            f"[{source['position']}] "
            f"{source['title']} — {source['domain']} — {source['url']}"
        )

This output is useful during development. Production jobs should write structured records to object storage, a database, or JSON Lines instead of relying on console output.

Build citation-level metrics

Once web_results is normalized, several useful measurements become simple.

Domain presence

Check whether a tracked domain appears anywhere in the source set.

def cites_domain(capture: dict, tracked_domain: str) -> bool:
    tracked = tracked_domain.lower().removeprefix("www.")
    return any(
        item["domain"] == tracked or item["domain"].endswith(f".{tracked}")
        for item in capture["citations"]
    )

Citation position

Track the first source position at which the domain appears. Keep presence and position separate: a missing domain should be None, not position zero.

def first_citation_position(capture: dict, tracked_domain: str):
    tracked = tracked_domain.lower().removeprefix("www.")
    positions = [
        item["position"]
        for item in capture["citations"]
        if item["domain"] == tracked or item["domain"].endswith(f".{tracked}")
    ]
    return min(positions) if positions else None

Source diversity

Count distinct domains rather than raw URLs. Several cited pages from one site represent a different source profile from the same number of pages spread across many domains.

Citation overlap

For two captures of the same prompt, compare the domain sets. The overlap reveals whether Perplexity's answer changed while leaning on the same sources or changed because the source set moved.

Keep comparisons scientifically useful

Perplexity answers can vary. The goal is not to force one fixed output; the goal is to preserve the inputs and measure the output honestly.

Use these controls:

1. Keep the prompt text byte-for-byte identical inside a series.
2. Pin the same country.
3. Keep web_search fixed.
4. Record the capture time and task_id.
5. Store the full source objects, not only their domains.
6. Compare several captures before treating a change as a trend.

Do not drop an answer because media_items or related_prompt is empty. Optional panels depend on the query.

Scale beyond one prompt

The request function should stay small. Scaling belongs in the orchestration layer:

• Read prompts from a versioned file or database table.
• Generate one job for each prompt and country pair.
• Limit concurrency to a value your account and workload support.
• Write each raw response before running transforms.
• Make normalization idempotent so it can be re-run from stored captures.

The Scrapeless Perplexity Scraper product page shows the current result surface, while the broader LLM scraper explainer places this workflow in a multi-engine monitoring system.

Conclusion

Scraping Perplexity answers reduces to one request and one platform adapter: send {actor: "scraper.perplexity", input: {prompt, country, web_search}}, read result_text for the answer, and normalize web_results into citation records.

From there, citation monitoring is standard data engineering. Preserve the raw response, normalize domains, track positions, and compare fixed prompt panels over time. You can create an API key in the Scrapeless dashboard.

FAQ

Does every Perplexity response contain citations?

No. The source panels depend on the query and answer path. Code should accept an empty web_results array.

Can I collect images and maps?

When Perplexity returns rich media, media_items can include media type, URLs, thumbnails, and location data. Treat those fields as optional.

Why should I store the prompt in every row?

The prompt is part of the measurement. Without the exact input, two answers cannot be compared reliably.

Is scraping Perplexity answers legal?

The actor collects publicly rendered answer content. Review applicable laws and Perplexity's terms, and consult counsel before collecting or redistributing data at scale.

---

Scrapeless accesses publicly available data and expects customers to follow applicable laws, platform terms, and privacy requirements. The examples in this article are for technical demonstration and should not be used to collect private, confidential, or restricted information.