DEV Community: Batulzii Ulziikhuu

Does Perplexity Mention Your Brand? Check AI Answers for Mentions, Position and Citations

Batulzii Ulziikhuu — Tue, 03 Nov 2026 13:34:46 +0000

Does Perplexity Mention Your Brand? How to Check AI Answers Systematically

More buyers now ask an AI assistant "what's the best X for Y?" instead of scrolling search results. If you do SEO or marketing, the obvious question is: does the answer mention us, where, and does it cite our site? Checking by hand once tells you little, because answers change. Checking the same questions on a schedule tells you a lot more.

This guide uses the AI Search Visibility Checker on Apify with the Perplexity Sonar API. It also supports ChatGPT (OpenAI API with web search) and Claude (Anthropic API with web search), but here we focus on Perplexity.

What you get per prompt

For every prompt × engine, one record with:

brandMentioned, brandPosition (rank of your first mention among the brands named), brandMentionCount
brandCited: whether one of your domains is among the sources the answer cites
competitorsMentioned and brandsMentionedInOrder
citations (sources the answer actually cites) and a simple sentiment, plus shareOfVoice
the answerText (truncated to maxAnswerChars in the output)

The last record is a type: "summary" with a visibility score per engine, mention rate, citation rate, average position, top competitors, top cited domains and prompts where only competitors appear.

Step 1: Get a Perplexity API key

From Perplexity's official quickstart:

Sign in to the Perplexity API Console and open the API Keys page.
Generate a new key. Keys belong to a project; see Perplexity's docs on projects.
Add billing. Perplexity's API is pay-as-you-go, no subscription required; check their pricing page for current rates.

Treat the key like a password: don't print it, paste it into logs or commit it.

Step 2: Apify token

Create an account at apify.com and copy your token from Settings → API & Integrations.

export APIFY_TOKEN=your_apify_token
export PERPLEXITY_API_KEY=your_perplexity_key

Step 3: Run it (JavaScript)

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('batuweb/ai-search-visibility-checker').call({
  prompts: ['best project management tool for small agencies', 'asana alternatives'],
  brandName: 'YourBrand',
  brandAliases: ['YourBrand App'],
  brandDomains: ['yourbrand.com'],
  competitors: ['Asana', 'Trello', 'Monday.com'],
  engines: ['perplexity'],
  perplexityApiKey: process.env.PERPLEXITY_API_KEY,
  countryCode: 'us',
  languageCode: 'en',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const r of items.filter((i) => i.type === 'result')) {
  console.log(r.prompt, '| mentioned:', r.brandMentioned, '| position:', r.brandPosition, '| cited:', r.brandCited);
}
const summary = items.find((i) => i.type === 'summary');
console.log(JSON.stringify(summary, null, 2));

Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("batuweb/ai-search-visibility-checker").call(run_input={
    "prompts": ["best project management tool for small agencies"],
    "brandName": "YourBrand",
    "brandDomains": ["yourbrand.com"],
    "competitors": ["Asana", "Trello"],
    "engines": ["perplexity"],
    "perplexityApiKey": os.environ["PERPLEXITY_API_KEY"],
})

for r in client.dataset(run["defaultDatasetId"]).iterate_items():
    if r.get("type") == "result":
        print(r["prompt"], r["brandMentioned"], r.get("brandPosition"), r["brandCited"])

curl

curl -X POST \
  "https://api.apify.com/v2/acts/batuweb~ai-search-visibility-checker/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"prompts\":[\"best project management tool for small agencies\"],\"brandName\":\"YourBrand\",\"brandDomains\":[\"yourbrand.com\"],\"competitors\":[\"Asana\",\"Trello\"],\"engines\":[\"perplexity\"],\"perplexityApiKey\":\"$PERPLEXITY_API_KEY\"}"

(Passing a secret on the command line can leave it in shell history; the client libraries above are the safer route.)

No key? The run returns a single free info record explaining how to start, and nothing is called or charged. Leave all brand fields empty to run a built-in demo (Semrush vs Ahrefs, Moz, Screaming Frog), which still needs an engine key.

What it costs

There are two separate bills:

Actor fee (Apify): pay per event, $0.008 per analysed AI answer on the Free plan, cheaper on paid Apify plans, plus a tiny start fee. Errors, empty or incomplete answers and the summary are free.
Perplexity API (Perplexity bills you directly): at their current pay-as-you-go rates. Search-backed requests can be billed on top of tokens, so check their pricing page.

So 10 prompts checked on Perplexity = 10 Actor events plus whatever Perplexity charges for 10 Sonar requests.

Tips

Use real buyer questions ("best X for Y", "X alternatives", "X vs Y"), not your brand name.
Add aliases (product names, old names) so every mention counts, and add your domains explicitly; they are not derived automatically.
Schedule a weekly run (Apify Schedules) and watch the trend instead of reacting to one answer.

Honest limits

Answers vary. The same prompt can give different answers on different runs. One run is a sample, not a verdict.
API ≠ app. This measures Perplexity's Sonar API answers. They are similar to, but not the same as, what a signed-in user sees in the Perplexity app (personalisation, model versions, UI features).
Sentiment is a simple lexicon around your mention, not a full NLP model.
The visibility score is a heuristic for tracking, not a ranking factor, and no tool can promise that an AI assistant will mention you.
Your use of the answers is subject to Perplexity's API terms.

AI Search Visibility Checker on the Apify Store
SEO Audit Toolkit: technical SEO crawler with monthly change monitoring.

Disclosure: I built the AI Search Visibility Checker Actor.

Monthly SEO Change Monitoring: Get a 'What Changed or Broke' Report From a Scheduled Audit

Batulzii Ulziikhuu — Tue, 03 Nov 2026 13:34:44 +0000

Monthly SEO Change Monitoring: "What Changed or Broke Since Last Month?"

A one-off SEO audit tells you what is wrong today. What agencies and site owners usually need is different: what changed since last time? Did a deploy break pages? Did someone rewrite a title? Did a fix actually stick?

This guide shows how to get exactly that from a scheduled audit with the SEO Audit Toolkit on Apify, using its free monitoring mode. There is also a ready-made Store example: Monitor website SEO changes.

How it works

Turn on compareWithPrevious and the Actor saves a snapshot of each audit in your own Apify account.

First run: saved as the baseline.
Every later run: adds a type: "changes" record to the dataset (also stored as CHANGES in the key-value store, plus a readable Markdown report as CHANGES_REPORT).

The changes record lists:

newly broken pages (new 404s/5xx or unreachable URLs), recovered pages and every status code change
new and fixed broken internal links
new and fixed issues per page and site-wide, and the issue count change per severity
title, meta description, noindex and canonical changes
site score and page score changes, new pages and pages not seen this run
new accessibility rule violations and ones no longer found (only when the accessibility scan is on in both runs, see limits below)

Monitoring costs nothing extra. You pay the normal per-page audit fee (see the Store page for current pricing).

Step 1: API token

Create an account at apify.com and copy your token from Settings → API & Integrations:

export APIFY_TOKEN=your_token

Step 2: Run with monitoring on (JavaScript)

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('batuweb/seo-audit-toolkit').call({
  startUrls: ['https://example.com'],
  crawlMode: 'sitemap',
  maxPages: 50,
  checkBrokenLinks: true,
  compareWithPrevious: true,
  monitoringKey: 'client-a', // optional: separate history per client/site
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
const changes = items.find((i) => i.type === 'changes');

if (changes?.status === 'baseline') {
  console.log('Baseline saved. Run again later to see changes.');
} else if (changes) {
  console.log(changes.headline);
  for (const p of changes.newlyBrokenPages) console.log('BROKEN', p.url, p.before, '→', p.after);
  for (const i of changes.newIssues) console.log('NEW', i.code, 'on', i.url);
  if (changes.checksNotCompared?.length) console.log('Not compared:', changes.checksNotCompared);
}

Step 3: Python

pip install apify-client

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("batuweb/seo-audit-toolkit").call(run_input={
    "startUrls": ["https://example.com"],
    "maxPages": 50,
    "compareWithPrevious": True,
    "monitoringKey": "client-a",
})

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
changes = next((i for i in items if i.get("type") == "changes"), None)

if changes and changes["status"] == "compared":
    print(changes["headline"])
    for name, c in changes["counts"].items():
        if c["totalCount"]:
            print(f"{name}: {c['totalCount']} (listed {c['listedCount']}, omitted {c['omittedCount']})")

Step 4: curl

curl -X POST \
  "https://api.apify.com/v2/acts/batuweb~seo-audit-toolkit/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":["https://example.com"],"maxPages":50,"compareWithPrevious":true,"monitoringKey":"client-a"}'

Step 5: Make it monthly with an Apify Schedule

The option itself does not schedule anything. To get a monthly report:

In Apify Console, open the Actor, enter the input above and save it as a task (one task per client or site works well).
Open Schedules → Create new, pick a monthly interval and add the task.
On the task, add an integration (email, Slack or a webhook) so you're notified when a run finishes. Point it at the CHANGES_REPORT Markdown if you want something human-readable to forward.

Keep the same start URLs and a similar maxPages each month, so the comparison is like for like.

What a changes record looks like

Shape of the record (values illustrative):

{
  "type": "changes",
  "status": "compared",
  "monitoringKey": "client-a",
  "previousRunAt": "…",
  "currentRunAt": "…",
  "headline": "Score 92 → 89 (-3); 1 newly broken page(s), 2 new broken link(s), 3 new / 1 fixed issue(s), 1 title and 0 meta description change(s), 2 new page(s), 0 page(s) not seen this run.",
  "newlyBrokenPages": [{ "url": "https://example.com/pricing/", "before": 200, "after": 404 }],
  "newIssues": [{ "url": "https://example.com/blog/post", "code": "TITLE_TOO_SHORT" }],
  "titleChanges": [{ "url": "https://example.com/", "before": "Old title", "after": "New title" }],
  "checksNotCompared": [],
  "counts": { "newIssues": { "totalCount": 3, "listedCount": 3, "omittedCount": 0 } }
}

Honest limits

A check is compared only if it ran in both runs. If checkBrokenLinks was on last month and off now, broken links are listed under checksNotCompared and never reported as "fixed". The same applies to PageSpeed results.
Accessibility changes appear only when checkAccessibility is on in both runs. A violation counts as "no longer found" only when the page had a complete scan both times; partial or skipped scans are listed as not compared. The Store example doesn't enable the accessibility scan, so turn it on yourself if you want it. An automated scan finds only part of accessibility barriers and says nothing about legal conformance.
Pages not audited this run (outside the page limit, or failed) never count as fixed.
Long lists are capped at 100 items in the record and 50 per section in the Markdown report, but the headline and counts always show true totals, with "…and N more" in the report.
It reads server-rendered HTML; content rendered only by client-side JavaScript may be missed by the SEO checks.
Snapshots live in a named key-value store seo-audit-monitoring in your own account. Use monitoringKey to keep separate histories.
Only audit sites you own or have permission to audit.

Disclosure: I built the SEO Audit Toolkit Actor.

How to Scan a Website for WCAG Accessibility Issues Automatically (JS, Python & curl)

Batulzii Ulziikhuu — Mon, 19 Oct 2026 10:56:44 +0000

How to Scan a Website for WCAG Accessibility Issues Automatically

Checking one page with a browser extension is easy. Checking the 40 pages of a client site every month, keeping the results and comparing them is not. This guide shows how to run an automated WCAG 2.x scan on many pages at once through an API and get the results back as JSON, with JavaScript, Python or plain curl.

I'll also be clear about the limits up front, because they matter more for accessibility than for almost anything else.

What an automated scan can and can't do

Automated rules are good at things a machine can decide reliably: an image without alternative text, a form field without a label, a button with no accessible name, a missing lang attribute, invalid ARIA, many colour-contrast problems.

They can't judge whether alt text actually describes the image, whether the reading order makes sense, whether a page works with a keyboard and a screen reader, or whether captions are correct. The axe-core maintainers estimate that automated testing finds on average about 57% of WCAG issues. The rest needs manual and assistive-technology testing.

So treat a scan as a fast first pass that helps you find and prioritise issues, not as an answer to "is this site accessible?". A result with zero violations does not mean a site is accessible.

Disclaimer. The tool below runs automated accessibility tests (based on the open-source axe-core® engine) and reports issues it can detect against WCAG 2.x success criteria. The results are not a conformity assessment, certification or proof of compliance with WCAG, EN 301 549, the European Accessibility Act, the German BFSG, the ADA or any other law, and not legal advice. Whether and how accessibility laws apply to your organisation must be assessed by a qualified professional.

axe-core® is a trademark of Deque Systems, Inc. in the US and other countries. The tool is not affiliated with or endorsed by Deque Systems.

A note on WCAG versions and the EAA

If you're looking at this because of the European Accessibility Act (in Germany the BFSG), one detail is worth knowing: the current harmonised standard, EN 301 549 V3.2.1, references WCAG 2.1 AA and has requirements beyond WCAG. A WCAG 2.2 scan is a useful technical check while you prepare, but it doesn't answer whether a site meets those rules.

Also, the default rule set runs the axe-core rules tagged wcag2a, wcag2aa, wcag21a, wcag21aa and wcag22aa. axe-core has no automated rules for the success criteria WCAG 2.2 added at Level A, so those need manual review too.

The tool: SEO Audit Tool with the accessibility check switched on

I maintain the SEO Audit Tool on Apify, a hosted crawler for technical SEO checks. It has an optional accessibility check: with checkAccessibility: true it opens the audited pages in headless Chrome, runs the unmodified axe-core rules and adds the results to each page record.

What you get per page:

violations: rule, impact, WCAG success criteria (such as 1.4.3), a short how-to-fix and up to 10 affected elements (CSS selector, failure summary and a snippet of at most 250 characters)
needsManualReview: checks the engine couldn't decide automatically, listed separately so they don't get lost or overstated
status: checked, partial (for example a stylesheet failed to load, so the page may look different to visitors) or skipped
a one-line disclaimer in English and German

The summary record lists the most common issues across all pages, and a Markdown report (English or German) is saved as ACCESSIBILITY_REPORT. There's deliberately no accessibility "score" and no pass/fail flag.

It's also polite: robots.txt is respected for every request the browser makes, pages are checked one at a time with at least one second between pages on the same host, and it never logs in or submits forms. Only scan sites you own or are authorised to test.

Run it from the browser

Open the ready-made example Check website accessibility (WCAG), replace the start URL with your site and click Start. Use 1024 MB memory: the browser scan works with 512 MB but takes about twice as long.

Run it with curl

curl -X POST "https://api.apify.com/v2/acts/batuweb~seo-audit-toolkit/run-sync-get-dataset-items?memory=1024" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": ["https://example.com"],
    "maxPages": 10,
    "checkBrokenLinks": false,
    "checkAccessibility": true,
    "accessibilityStandard": "wcag22aa",
    "accessibilityLanguage": "en"
  }'

You get an array of records: one per audited page (each with an accessibility object), error records for unreachable URLs, and a summary record at the end.

JavaScript (Node.js)

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('batuweb/seo-audit-toolkit').call(
  {
    startUrls: ['https://example.com'],
    maxPages: 20,
    checkAccessibility: true,
    accessibilityMaxPages: 20,
    accessibilityStandard: 'wcag21aa', // the WCAG version EN 301 549 V3.2.1 references
  },
  { memory: 1024 },
);

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const page of items.filter((i) => i.type === 'page')) {
  const a = page.accessibility;
  if (!a || a.status === 'skipped') continue;
  console.log(`${page.url} [${a.status}] violations: ${a.counts.violations}, manual review: ${a.counts.needsManualReview}`);
  for (const v of a.violations) console.log(`  - ${v.ruleId} (${v.impact}, WCAG ${v.wcagCriteria.join(', ')}): ${v.help} x${v.elementCount}`);
}

Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("batuweb/seo-audit-toolkit").call(
    run_input={
        "startUrls": ["https://example.com"],
        "maxPages": 20,
        "checkAccessibility": True,
        "accessibilityIncludeHtmlSnippets": False,  # keep page content out of the results
    },
    memory_mbytes=1024,
)

items = client.dataset(run["defaultDatasetId"]).list_items().items
summary = next(i for i in items if i.get("type") == "summary")["accessibility"]
print("Pages checked:", summary["pagesChecked"], "partial:", summary["pagesPartiallyChecked"])
for rule in summary["topViolations"]:
    print(f'{rule["ruleId"]}: {rule["pages"]} page(s), {rule["elements"]} element(s)')
for rule in summary["needsManualReview"]:
    print(f'Needs manual review: {rule["ruleId"]} on {rule["pages"]} page(s)')

A real example

I ran it on a small recipe site of mine, cookmushroom.com (10 pages, 1024 MB, about 100 seconds). All 10 pages were fully checked, and there were 0 automated violations. That sounds great, but the useful part was the manual-review list:

color-contrast (WCAG 1.4.3) on all 10 pages: the engine couldn't work out the background colour because of a CSS gradient, so a person has to check the contrast.
link-in-text-block (WCAG 1.4.1) on 5 pages: links inside paragraphs may only be distinguishable by colour.

That's exactly the kind of result to read carefully: "0 violations" here means "nothing the machine could prove wrong", not "accessible".

Tips

Pick the rule set on purpose. wcag21aa matches the WCAG version that EN 301 549 V3.2.1 references; wcag22aa adds the WCAG 2.2 AA rules.
Look at partial pages. If a stylesheet or script didn't load, contrast and visibility results can differ from what visitors see.
Schedule it. Run it monthly with an Apify schedule and compare the summaries to catch regressions after redesigns.
Turn snippets off (accessibilityIncludeHtmlSnippets: false) if pages can contain personal data. Only CSS selectors are kept then.
Follow up manually. Use the scan to find the obvious issues fast, then do keyboard, zoom and screen-reader checks on your key templates.

Pricing

You pay the normal SEO audit fee of $0.004 per page (cheaper on paid Apify plans). The accessibility check is free during launch. From 3 November 2026 it costs $0.004 per fully checked page on the Free plan, down to $0.002 on higher plans. Partial and skipped pages are never charged, and you can cap the cost of any run with Maximum cost per run.

Automated testing finds only part of all accessibility barriers. Results help you find and prioritise issues; they are not a conformity assessment, certification or legal advice.

How to Take Full-Page Website Screenshots and PDFs via API (No Puppeteer Setup)

Batulzii Ulziikhuu — Mon, 19 Oct 2026 08:13:52 +0000

Taking one screenshot with Puppeteer or Playwright is easy. Taking thousands reliably is not. You end up managing Chromium versions in Docker, memory limits, lazy-loaded images that never appear, cookie banners covering half the page, timeouts, retries and a queue.

This guide shows how to skip that and call a hosted screenshot API instead: send a list of URLs, get back PNG, JPEG, WebP or PDF files plus a JSON record per URL. I'll show plain HTTP, JavaScript and Python, and I'll be upfront about the limits.

When a screenshot API makes sense

Visual monitoring: a daily capture of your landing pages to spot broken layouts.
SEO and client reports: desktop and mobile above-the-fold captures side by side.
Archiving: save articles, documentation or your own pages as PDF.
Link previews and thumbnails for dashboards, directories or internal tools.
QA: check how a release looks on a phone, a tablet and a large screen.

If you only need a single screenshot once a month, a local Playwright script is fine. Once you're dealing with batches, schedules or several devices, a hosted service saves a lot of maintenance.

The tool: Website Screenshot & PDF API on Apify

Website Screenshot & PDF API is an Apify Actor, a hosted program you can call through the Apify API. What it does:

Full-page or viewport captures (full page up to a configurable height, default 10,000 px)
Device presets: desktop 1366×768, laptop 1920×1080, tablet (iPad, 2×), mobile (iPhone 390×844, 2×) or a custom size
Formats: PNG, JPEG, WebP and PDF (A4, Letter, Legal or A3, portrait or landscape, with backgrounds)
Cookie banners hidden with CSS for about 45 common consent managers. Nothing is clicked or accepted.
Lazy-image scrolling, wait for a selector, extra delay, or network-idle waiting
Element capture: only header, #pricing and so on
Dark mode emulation
Batch: up to 5,000 URLs per run, up to 10 pages in parallel

Respectful by default: it checks each site's robots.txt, including the targets of redirects, and skips disallowed URLs. It only captures public websites (localhost and private-network addresses are blocked), and it doesn't try to get around logins, paywalls, CAPTCHAs or bot protection.

Pricing

You pay per successful capture:

Event	Free plan	Higher Apify plans
Screenshot (PNG/JPEG/WebP)	$0.002	down to $0.0008
PDF	$0.003	down to $0.0015

There's also a tiny start fee per run ($0.00005 per GB of memory). Failed, timed-out, robots-blocked and invalid URLs are free. Each one gets an error record in the dataset instead of a charge. Pages that answer with an HTTP error (4xx/5xx) are free error records too, unless you explicitly turn on captureErrorPages. Apify's free plan includes monthly credits, which covers a lot of test screenshots.

Apify also has its own Screenshot URL Actor with URL lists, PDF output, scrolling and selectors to hide. Both work well, so pick whichever fits your workflow. This one focuses on device presets, WebP, cookie-banner hiding, element capture and per-URL error codes with pay-per-success pricing.

Step 1: Get an Apify API token

Create a free account at apify.com.
Open Settings → API & Integrations and copy your API token.
Store it as an environment variable: export APIFY_TOKEN=...

Step 2: One request with curl (any language)

The synchronous endpoint runs the Actor and returns the dataset in a single HTTP call:

curl -X POST \
  "https://api.apify.com/v2/acts/batuweb~website-screenshot/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [{ "url": "https://crawlee.dev" }],
    "format": "png",
    "fullPage": true,
    "device": "desktop"
  }'

The response is a JSON array with one item per URL. imageUrl is a direct link to the file:

[
  {
    "success": true,
    "url": "https://crawlee.dev",
    "finalUrl": "https://crawlee.dev/",
    "status": 200,
    "title": "Crawlee · Build reliable crawlers. Fast.",
    "format": "png",
    "device": "desktop",
    "viewportWidth": 1366,
    "viewportHeight": 768,
    "fullPage": true,
    "width": 1366,
    "height": 2874,
    "truncated": false,
    "pageCount": null,
    "fileSize": 330185,
    "imageUrl": "https://api.apify.com/v2/key-value-stores/<storeId>/records/0001-crawlee-dev-0d35627b.png?signature=...",
    "key": "0001-crawlee-dev-0d35627b.png",
    "errorCode": null,
    "error": null,
    "capturedAt": "2026-10-19T08:11:52.257Z"
  }
]

This works from n8n, Make, Zapier, Google Apps Script or a cron job.

Step 3: Full-page and mobile screenshots with JavaScript

npm install apify-client

import { ApifyClient } from 'apify-client';
import { writeFile } from 'node:fs/promises';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('batuweb/website-screenshot').call({
  urls: [
    { url: 'https://crawlee.dev' },
    { url: 'https://lightertorch.com' },
  ],
  format: 'webp',          // png | jpeg | webp | pdf
  quality: 80,             // JPEG/WebP only
  device: 'mobile',        // desktop | laptop | tablet | mobile | custom
  fullPage: false,         // only the visible viewport ("above the fold")
  hideCookieBanners: true,
  maxConcurrency: 3,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const item of items) {
  if (!item.success) {
    console.log('skipped (free):', item.url, item.errorCode);
    continue;
  }
  const res = await fetch(item.imageUrl);
  await writeFile(item.key, Buffer.from(await res.arrayBuffer()));
  console.log('saved', item.key, item.width + 'x' + item.height);
}

Want only one section of a page? Add selector: '#pricing' (image formats only). For JavaScript-heavy pages, use waitUntil: 'networkidle' or waitForSelector: '.chart' together with delayMs: 1000.

Step 4: Website to PDF with Python

pip install apify-client requests

import os
import requests
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("batuweb/website-screenshot").call(run_input={
    "urls": [
        {"url": "https://docs.apify.com/platform/actors"},
        {"url": "https://crawlee.dev/js/docs/quick-start"},
    ],
    "format": "pdf",
    "pdfPaper": "a4",            # a4 | letter | legal | a3
    "pdfLandscape": False,
    "pdfPrintBackground": True,
    "scrollToBottom": True,      # load lazy images before printing
    "maxFullPageHeight": 10000,  # long pages are cut here (truncated: true)
})

for item in client.dataset(run.default_dataset_id).iterate_items():
    if not item["success"]:
        print("free error record:", item["url"], item["errorCode"])
        continue
    with open(item["key"], "wb") as f:
        f.write(requests.get(item["imageUrl"], timeout=60).content)
    print("saved", item["key"], item.get("pageCount"), "pages")

The files also stay in the run's key-value store, under the key shown in each record, for the store's normal retention period. So you can fetch them later with client.key_value_store(run.default_key_value_store_id).get_record(key).

(This uses the current apify-client 3.x for Python, where call() returns a Run object. With the older 1.x client, use run["defaultDatasetId"] instead.)

Step 5: Batch screenshots on a schedule

For visual monitoring, save the input as an Apify task and add a schedule (for example every morning at 07:00). Each run creates a dataset you can export as JSON, CSV or Excel, or push to a webhook, Google Drive or Slack through Apify integrations.

Two settings help control cost in big batches:

maxScreenshots: stop after N successful captures.
The run's max total charge: the Actor stops cleanly once your budget is reached.

You can try ready-made examples without writing any code:

Error codes you may see (all free)

`errorCode`	Meaning
`ROBOTS_DISALLOWED`	The URL or its redirect target is disallowed by robots.txt, or robots.txt couldn't be fetched (treated as "disallow all")
`HTTP_403`, `HTTP_404`, ...	The page answered with an HTTP error
`TIMEOUT`, `DNS_ERROR`, `TLS_ERROR`, `CONNECTION_ERROR`	The page couldn't be loaded within `navigationTimeoutSecs` (after retries)
`BLOCKED_PRIVATE_ADDRESS`	The URL or a redirect points to localhost, a private network or a cloud metadata address
`SELECTOR_NOT_FOUND`	Your `selector` or `waitForSelector` didn't match a visible element
`INVALID_URL`, `TOO_MANY_REDIRECTS`	Exactly what it says

Honest limitations

robots.txt is a courtesy signal, not permission. It doesn't replace a site's terms of use, copyright or privacy law. Only capture pages you're allowed to: your own sites, clients' sites with their OK, or public pages whose terms allow it. Using this tool doesn't give you legal clearance for any particular use.
No login, paywall or CAPTCHA bypass. Sites that block automated browsers return an error (free). A block page served with HTTP 200 looks like a normal page and is captured as rendered.
Height limits: full-page captures and PDFs are cut at maxFullPageHeight (max 16,000 px) and marked truncated: true. WebP is also limited to 16,383 pixels per side.
Cookie banners from less common consent tools may still show. Add their selectors to selectorsToHide.
If a site's robots.txt is down or rate-limited, that site is skipped for the run. Retry later.

FAQ

Do I need to install Chromium or Puppeteer? No. The browser runs on Apify, so you only send HTTP requests.

How much do 1,000 screenshots cost? About $2 on the Apify Free plan, and down to $0.80 on higher plans, plus a negligible start fee. Failed URLs cost nothing.

Can it capture pages behind a login? No, only publicly reachable pages.

Can AI agents use it? Yes. Like any Apify Actor, it can be called through the Apify MCP server or the REST API.

Related tools

SEO Audit Tool – Site Crawler, Broken Links & Meta Checker: pair visual captures with a technical SEO check of the same pages.
Google Trends Scraper: interest over time, regions and related queries as JSON.

Disclosure: I built the Website Screenshot & PDF API Actor.

Free SEO Audit Crawler API: Automate Technical SEO Checks with JavaScript or Python

Batulzii Ulziikhuu — Sun, 18 Oct 2026 18:56:07 +0000

Desktop crawlers like Screaming Frog are excellent, but they're manual: open the app, crawl, export, repeat. If you manage several sites, run an agency, or want SEO checks in CI after every deploy, you want an SEO audit crawler you can call as an API and get JSON back.

This guide shows how to do that with the SEO Audit Toolkit on Apify, from JavaScript, Python and curl, and how to turn the results into automated alerts.

What the audit checks

For every crawled page:

Title and meta description: missing, too short/long, duplicates across the site
H1: missing or multiple
Canonical, noindex / robots meta (including none), hreflang
Structured data (JSON-LD/schema.org) and Open Graph tags
Image alt text, word count / thin content
Broken internal and external links (HTTP status checks)
HTTP errors and redirect problems

Site-wide:

robots.txt parsing (including "blocks all crawlers" detection)
sitemap.xml validation (index sitemaps, URL counts, invalid XML)
Tech stack detection
Optional Google PageSpeed / Lighthouse scores with your own API key

You get a 0–100 site score plus a prioritized issue list with example URLs, so you know what to fix first.

Is it free?

Honestly: almost. You pay per page audited ($0.004 per page on Apify's Free plan, down to $0.0015 on paid plans) plus a tiny start fee. A 50-page audit is about $0.20. Apify's free plan includes monthly credits, so small sites and testing effectively cost nothing out of pocket. The summary record and unreachable-URL errors are free, and you can cap spend per run.

Step 1: API token

Create an account at apify.com, copy your token from Settings → API & Integrations:

export APIFY_TOKEN=your_token

Step 2: Run an audit with JavaScript

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('batuweb/seo-audit-toolkit').call({
  startUrls: ['https://example.com'],
  crawlMode: 'sitemap',   // or crawl links from the start URL
  maxPages: 50,
  checkBrokenLinks: true,
  maxLinkChecks: 200,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
const summary = items.find((i) => i.type === 'summary'); // summary is the last record
console.log('Score:', summary.score);
for (const issue of summary.prioritizedIssues.slice(0, 10)) {
  console.log(`${issue.severity.padEnd(8)} ${issue.code} (${issue.pages} pages) e.g. ${issue.exampleUrls[0]}`);
}

Step 3: Python

pip install apify-client

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("batuweb/seo-audit-toolkit").call(run_input={
    "startUrls": ["https://example.com"],
    "maxPages": 100,
})

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
summary = next(i for i in items if i.get("type") == "summary")
pages = [i for i in items if i.get("type") == "page"]

print("Site score:", summary["score"], "| pages:", summary["pagesAudited"])
print(summary["issuesBySeverity"])

Step 4: curl / no-code

curl -X POST \
  "https://api.apify.com/v2/acts/batuweb~seo-audit-toolkit/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":["https://example.com"],"maxPages":20}'

The same endpoint works from n8n, Make, Zapier or Google Sheets (Apps Script UrlFetchApp).

Output example (summary record)

{
  "type": "summary",
  "score": 97,
  "pagesAudited": 10,
  "pagesFailed": 0,
  "issuesBySeverity": { "critical": 0, "high": 1, "medium": 2, "low": 11 },
  "prioritizedIssues": [
    { "code": "BROKEN_INTERNAL_LINKS", "severity": "high", "pages": 1, "exampleUrls": ["https://example.com/about/"] },
    { "code": "TITLE_TOO_SHORT", "severity": "medium", "pages": 2, "exampleUrls": ["https://example.com/contact/"] }
  ],
  "robotsTxt": { "https://example.com": { "status": 200, "blocksAllCrawlers": false } },
  "sitemaps": [{ "url": "https://example.com/sitemap-0.xml", "status": 200, "valid": true, "urlCount": 51 }]
}

Each page also gets its own record with all checks and issues, which is handy for spreadsheets.

Automation ideas

SEO regression check in CI: run after deploy and fail the pipeline if the score drops or a critical issue appears:

if (summary.issuesBySeverity.critical > 0 || summary.score < 85) process.exit(1);

Weekly client reports: an Apify schedule per client site, with a webhook to Slack or email.
Broken-link monitoring across a portfolio of sites.
Pre-migration baselines: audit before and after a redesign and diff the page records.
AI agents: call it via the Apify MCP server so an assistant can audit a URL on request.

Honest limitations

It is an HTML crawler, not a headless browser. Content rendered only by client-side JavaScript may be missed.
PageSpeed/Lighthouse metrics need your own free Google API key and cost extra per page.
The score is a heuristic for prioritization, not a Google ranking factor.
Only audit sites you own or have permission to audit. Private/internal network addresses are blocked by design.

FAQ

How is this different from Screaming Frog? Screaming Frog is a powerful desktop app. This is a lightweight cloud API for automation, scheduling and integrations. Many people use both.

How many pages can I audit? Set maxPages. Cost scales linearly, and you can set a maximum cost per run.

Can I export CSV? Yes, any Apify dataset downloads as CSV, Excel or JSON.

Related tools

Google Trends Scraper: find rising topics and queries to target.

Disclosure: I built the SEO Audit Toolkit Actor.

Google Trends API in 2026: How to Get Trends Data Programmatically (JS & Python)

Batulzii Ulziikhuu — Sun, 18 Oct 2026 18:49:06 +0000

If you have ever searched for a "Google Trends API", you probably found three things: an alpha API from Google with a waitlist, the unofficial pytrends library that keeps breaking with HTTP 429 errors, and a lot of scraping tutorials that stop working after a week.

This guide shows a practical alternative: calling a hosted Google Trends scraper as an API. You send keywords, you get back clean JSON with interest over time, interest by region, related queries and related topics. I'll show JavaScript, Python and plain curl, and I'll be honest about the limits.

Why getting Google Trends data is harder than it looks

Google Trends has no generally available, self-serve public API. The website itself loads data from internal endpoints that need a fresh token per widget, and Google rate-limits those endpoints aggressively. That's why:

pytrends scripts often fail with 429 Too Many Requests after a few calls, especially from cloud servers.
Running your own scraper means managing cookies, proxies, retries and backoff.
Many hosted scrapers fail a noticeable share of runs. On the Apify Store the official Trends actor's public stats have shown a failure rate above 20% at times.

What you actually want is: keywords in, JSON out, it just works, and you don't pay for failures.

The tool: Google Trends Scraper on Apify

Google Trends Scraper is an Apify Actor (a hosted, API-callable scraper). It supports:

Interest over time for up to 5 compared keywords (e.g. "coffee, tea")
Interest by region: country, region/state, metro or city
Related queries and related topics, both top and rising (including "Breakout")
Search types: Web, News, YouTube, Images and Google Shopping
Any country (geo), category and time range, including custom date ranges
Automatic retries, session rotation and residential proxy fallback (proxy cost included)

Pricing is pay-per-result: about $0.002 per search that returns data (cheaper on higher Apify plans) plus a tiny start fee. Empty or failed searches are not charged. Apify's free plan includes monthly credits, which covers a lot of test searches.

Step 1: Get an Apify API token

Create a free account at apify.com.
Go to Settings → API & Integrations and copy your API token.
Store it as an environment variable: export APIFY_TOKEN=...

Step 2: Call the Google Trends API with JavaScript

npm install apify-client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('batuweb/google-trends-scraper').call({
  searchTerms: ['coffee, tea', 'matcha'], // each line = one search, commas = comparison
  geo: 'DE',                               // '' = worldwide
  timeframe: 'today 12-m',
  includeInterestOverTime: true,
  includeInterestByRegion: true,
  includeRelatedQueries: true,
  includeRelatedTopics: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const item of items.filter((i) => i.type === 'result')) {
  console.log(item.input, item.averageInterest);
}

Step 3: Or with Python

pip install apify-client

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("batuweb/google-trends-scraper").call(run_input={
    "searchTerms": ["web scraping", "coffee, tea"],
    "geo": "US",
    "timeframe": "today 5-y",
    "gprop": "youtube",   # web | news | youtube | images | froogle (Shopping)
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item.get("type") != "result":
        continue
    print(item["input"], item["averageInterest"])
    for q in item.get("relatedQueries", {}).get("coffee", {}).get("rising", [])[:5]:
        print("  rising:", q["query"], q.get("formattedValue"))

Loading the timeline into pandas is one line:

import pandas as pd
points = item["interestOverTime"]["points"]
df = pd.DataFrame([{"date": p["date"], **p["values"]} for p in points])

Step 4: Plain HTTP (any language, no SDK)

The synchronous endpoint runs the scraper and returns the dataset items in one request:

curl -X POST \
  "https://api.apify.com/v2/acts/batuweb~google-trends-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchTerms":["chatgpt, gemini"],"geo":"","timeframe":"today 3-m"}'

This works from n8n, Make, Zapier, Google Apps Script or a cron job.

What the output looks like

{
  "type": "result",
  "input": "coffee, tea",
  "terms": ["coffee", "tea"],
  "geo": "DE",
  "timeframe": "today 12-m",
  "averageInterest": { "coffee": 24, "tea": 19 },
  "interestOverTime": { "points": [{ "date": "2025-10-12T00:00:00.000Z", "values": { "coffee": 24, "tea": 19 } }] },
  "interestByRegion": { "resolution": "REGION", "regions": [{ "geoCode": "DE-HE", "geoName": "Hesse", "values": { "coffee": 65, "tea": 35 } }] },
  "relatedQueries": { "coffee": { "top": [{ "query": "coffee machine", "value": 100 }], "rising": [] } },
  "trendsUrl": "https://trends.google.com/trends/explore?q=coffee%2Ctea&date=today+12-m&geo=DE"
}

If one section fails (for example related queries), you still get the other sections plus an explicit error field rather than losing the whole result.

Practical use cases

SEO & content planning: find rising queries before competitors write about them.
Seasonality: pull 5-year timelines to plan campaigns and stock levels.
Market research: compare brands or products across countries and regions.
Dashboards: schedule a weekly run and push results into Google Sheets or BigQuery.
AI agents: the Actor can be called through the Apify MCP server.

Honest limitations

Values are relative (0–100), not absolute search volumes. That is how Google Trends works; no scraper can change it.
Comparisons are limited to 5 keywords per search, like on the Trends website.
Very low-volume keywords may return no data. You aren't charged for those.
Google can change its internal endpoints. The Actor is monitored weekly, but this is still scraping, not an official SLA-backed API.
Check Google's terms and your local laws for your use case. The data is public, aggregated and anonymous.

FAQ

Is there an official Google Trends API? Google has announced an alpha Trends API with limited access. For most developers, a hosted scraper or pytrends is still the practical option.

Why not just use pytrends? It's fine for occasional local use. On servers, it hits rate limits quickly and you have to manage proxies yourself.

How much does 1,000 searches cost? Around $2 on the Free plan, less on paid plans, with no charge for empty results.

Related tools

SEO Audit Toolkit: audit titles, meta descriptions, broken links, sitemap and robots.txt for the pages you build around rising topics.

Disclosure: I built the Google Trends Scraper Actor.