DEV Community: Tidy Tools

Turn a website list into a B2B contact sheet with Apify and its API

Tidy Tools — Fri, 23 Oct 2026 04:31:18 +0000

A spreadsheet of company websites is a starting point, not yet a usable B2B contact list. It rarely tells you which address appeared on the official site, whether the domain accepts mail, whether a phone number came from the site or a directory, or which rows need a second look. Copying a single address from each home page also misses the contact, support and legal pages where businesses often publish it.

This walkthrough takes a list of public company websites through Website Email Extractor, reviews the results, and exports a sheet that a sales operations person can actually inspect. It shows both the Apify Console route and an API route. The evidence is a five-site run made on October 23, 2026; because websites change, treat its yield as a sample rather than a promise for your list.

Prepare a list worth running

Start with the company's own domain, one per line. Remove directory listing pages, social profiles and obvious duplicates before you run anything. If your source is a Google Maps export, keep the place ID and website column together: you will want to join the results back to the source. This Actor reads websites from a Maps dataset, but it does not scrape Google Maps itself.

For a first batch, use 10 to 20 sites. Check that you are allowed to collect and use the published contact details for your purpose. A public address is not a promise that someone wants unsolicited mail. Shared mailboxes such as press@ or support@ are useful for routing and research; they are not automatically sales inboxes.

The core input is deliberately small:

{
  "domains": [
    "katzsdelicatessen.com",
    "levainbakery.com",
    "tartinebakery.com",
    "stumptowncoffee.com",
    "bluebottlecoffee.com"
  ],
  "requiredContact": "email",
  "maxPagesPerSite": 4
}

This is exactly the input of the run shown below. maxPagesPerSite caps how far the Actor looks beyond a site's home page. The default is eight pages; four is a reasonable first pass for a small sample. requiredContact: "email" makes phone-only and social-only rows free while still returning them for review. Remove it if phones or social profiles are useful outcomes for you.

Run it in Apify Console

Open the Store page, choose Try for free, and paste your sites into Websites or company names. In the JSON editor you can use the object above. Set a maximum cost in Apify's run options if this is a new source list. Start the run, then inspect the Output table and the run's SUMMARY record before exporting.

Look at three columns first: bestEmail, charged, and errorType (present on failed rows). bestEmail is the selected address, not the only address. emails is an array of address strings; emailDetails holds each address's source page and checks. charged tells you whether this website triggered the per-site event. errorType explains a free row that was blocked, unreachable or invalid. A contact form without an email may still be useful, but it does not become an email lead.

The output can be downloaded as CSV, Excel or JSON. The Output tab also offers a Leads CSV view and a Contact coverage report; the report is saved as CONTACT_REPORT in the run's key-value store. The coverage report separates sites read this run, sites with email, sites without email, failures and rows reused from an earlier dedupe store. That separation matters: a sheet with 100 input rows is not evidence of 100 reachable prospects.

Actual example output

Run ID y5Kktfd6mTAnRSuDE used the five-domain input above on October 23, 2026. It finished in about 7 seconds. Its SUMMARY record reported 5 websites read, 4 with email, 4 with a phone number, 5 with at least one social profile, 0 failures, and 4 charged websites. One site had nothing billable under requiredContact: "email" and was returned free.

Input	`bestEmail`	Emails found	Pages checked	`charged`
katzsdelicatessen.com	press@katzsdelicatessen.com	1	4	true
levainbakery.com	media@levainbakery.com	1	4	true
tartinebakery.com	9th.ave@tartinebakery.com	18	3	true
stumptowncoffee.com	info@stumptowncoffee.com	1	4	true
bluebottlecoffee.com	(none)	0	4	false

Here is the Levain Bakery row from that dataset, reduced to the fields this workflow uses (its long phone list and the pagesChecked details are omitted):

{
  "input": "levainbakery.com",
  "finalUrl": "https://levainbakery.com/",
  "organizationName": "Levain Bakery",
  "bestEmail": "media@levainbakery.com",
  "bestEmailType": "role_company",
  "bestEmailHasMx": true,
  "emails": ["media@levainbakery.com"],
  "primaryPhone": "+18779322161",
  "contactFormUrl": "https://levainbakery.com/pages/get-in-touch",
  "emailDetails": [
    {"email": "media@levainbakery.com", "page": "https://levainbakery.com/pages/get-in-touch",
     "method": "mailto", "sameDomain": true, "syntaxValid": true, "hasMx": true, "category": "role_company"}
  ],
  "charged": true
}

The address came from the Get in touch page, not the home page, which is why reading a few contact pages per site is worth it. It is a media mailbox: useful for routing, not proof of a buyer's inbox.

The other rows show why the review step matters. Tartine's locations page produced 18 addresses, including shop mailboxes on separate Korean and manufactory domains, and the selected bestEmail was one shop's address even though info@tartinebakery.com was also found. For an account-level sheet you would probably pick the general address by hand. Blue Bottle returned four social profiles and a support help-center page but no email, so with requiredContact: "email" its row cost nothing and stays visible as "no email found".

hasMx: true says that a domain publishes mail servers. It does not prove that a particular mailbox exists or will accept your message. Four out of five is a property of this small, hand-picked sample, not an accuracy figure, and it tells you nothing about reply rates. Keep the missing-email rows in your sheet instead of quietly dropping them from your denominator.

Run the same job through the API

Apify's API uses the same Actor and JSON input. Keep your token in an environment variable; do not put it in a published script or a shared URL. This Python example starts a run, waits for its status, and downloads its dataset as JSON. It uses the same input as the run above, which was started through the same API endpoint.

import os
import time
import requests

api = "https://api.apify.com/v2"
headers = {"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"}
actor = "tidytools~website-contact-extractor"
payload = {
    "domains": ["katzsdelicatessen.com", "levainbakery.com", "tartinebakery.com",
                "stumptowncoffee.com", "bluebottlecoffee.com"],
    "requiredContact": "email",
    "maxPagesPerSite": 4,
}

response = requests.post(
    f"{api}/acts/{actor}/runs",
    headers=headers, json=payload, timeout=30,
)
response.raise_for_status()
run = response.json()["data"]
while run["status"] in ("READY", "RUNNING"):
    time.sleep(5)
    response = requests.get(f"{api}/actor-runs/{run['id']}", headers=headers, timeout=30)
    response.raise_for_status()
    run = response.json()["data"]
if run["status"] != "SUCCEEDED":
    raise RuntimeError(f"Run {run['id']} ended as {run['status']}")
response = requests.get(
    f"{api}/datasets/{run['defaultDatasetId']}/items",
    headers=headers, params={"format": "json", "clean": "true"}, timeout=30,
)
response.raise_for_status()
rows = response.json()
for row in rows:
    print(row.get("domain"), row.get("bestEmail"), row.get("charged"), row.get("errorType"))

For production use, save the run ID before waiting so a network interruption does not make you start and pay for the whole batch again. Fetch the dataset for that ID when the connection returns. Set Apify's maximum charge per run as another guard, and test your field mapping on a small batch before sending thousands of domains.

Turn rows into a reviewable sheet

Export one row per website first. Keep the input domain, normalized website, business name, bestEmail, all email addresses, source page, phone, social links, charged, and errorType. If your CRM needs one row per address, expand the emails array in a separate step while retaining the website key. Do not mistake 21 addresses for 21 companies: the example run covered 5 sites, of which 4 had email, and 18 of those addresses came from one bakery chain.

Review role and personal addresses separately. Check the source URL when an address looks generic or unrelated to the company. Deduplicate on normalized company website before importing to a CRM, and make it easy to remove or refresh stale contacts. Domain-level MX checking helps catch obvious dead domains but does not confirm individual delivery. If you need mailbox-level evidence, use a process designed for that and describe what it actually checks.

This workflow works especially well as a recurring enrichment step for account lists that already have official websites. It is less helpful when your starting point is only a company name with no location: a name can match several businesses. Add city or address data before asking the Actor to resolve a name, and manually inspect any low-confidence matches.

Pricing explanation

The current site-processed price in store_meta.py is $0.002 per charged website on Apify Free and Bronze plans: $2 per 1,000. Silver is $0.0018 and Gold or higher is $0.0016. There is no start event. With the default requiredContact: "any", a site is charged if at least one email, phone or social profile is found. With "email", it is charged only if an email is found. Unreachable, invalid, duplicate and no-contact rows are free.

Apify recorded site-processed: 4 for run y5Kktfd6mTAnRSuDE, so the run cost 4 × $0.002 = $0.008 at the Free/Bronze price; the Blue Bottle row was free. Had the same input used the default "any", Blue Bottle's social profiles would have made it billable: 5 × $0.002 = $0.010. That second number is a calculation, not a measured charge. A list of 1,000 sites costs at most $2.00 at Free/Bronze prices, and less when some sites have no email, so set a maximum charge before a large batch and check the Pricing tab.

If you have a domain list, try Website Email Extractor on Apify with 10 sites first. For the wider set of contact, validation and company-data options, see the TidyTools company-data directory. I built this Actor; the run above is an example of its output, not a claim that every list will produce the same yield.

Recover deleted pages and old URLs with the Wayback Machine, in bulk

Tidy Tools — Tue, 13 Oct 2026 02:03:35 +0000

Three situations where I have needed the Wayback Machine, and where its website is the wrong tool:

After a site migration, traffic dropped. Which old URLs used to exist, and which of them now 404 without a redirect?
An old blog post has 40 outbound links, and a third of them are dead. Each needs an archive link instead.
An expired domain is for sale with "20 years of history". Was it a real site for 20 years, or a parked redirect for the last ten?

The Internet Archive has a public index API (the CDX server) that answers all of these. It is free, generous and also slow and easy to overload: heavily archived pages can take 30–60 seconds per query, and it starts refusing at about 24 queries a minute. So I wrapped it in an Apify Actor, Wayback Machine Scraper, that queues politely, retries, de-duplicates, and returns clean rows. Here is how each job looks.

Job 1: every URL a site ever had

Mode urls lists every page the Wayback Machine has captured for a domain, a host or a path, with the first capture date:

{
  "urls": ["https://blog.example.com/"],
  "mode": "urls",
  "matchType": "prefix",
  "fromDate": "2020",
  "maxResultsPerInput": 5000
}

A real row for python.org:

{
  "type": "url",
  "input": "python.org",
  "url": "http://www.python.org/1.5/",
  "path": "/1.5/",
  "firstCapturedAt": "1998-01-19T01:50:21Z",
  "statusCode": 200,
  "mimeType": "text/html",
  "firstSnapshotUrl": "https://web.archive.org/web/19980119015021/http://www.python.org:80/1.5/",
  "latestSnapshotUrl": "https://web.archive.org/web/http://www.python.org/1.5/",
  "charged": true
}

By default you get only successful HTML pages, and http://example.com:80/a/ and https://www.example.com/a count once. That is what you want for a redirect map: export the path column, diff it against your current sitemap, and every path that is in the archive but not on the new site is a candidate for a 301. Add urlPattern (a regex) to focus on, say, /blog/ or /products/.

Job 2: an archive link for every dead link

Mode closest returns the snapshot nearest to a date (or the latest one) for each URL you paste:

{
  "urls": ["https://www.nasa.gov/missions/", "https://example.com/old-page"],
  "mode": "closest"
}

With targetDate and includeContent, you also get the archived page as Markdown, which is handy when you want to quote what a page said rather than just link to it:

{
  "type": "closest",
  "input": "https://www.nasa.gov/missions/",
  "targetDate": "2015-06-01T00:00:00Z",
  "capturedAt": "2015-06-12T18:50:45Z",
  "statusCode": 200,
  "snapshotUrl": "https://web.archive.org/web/20150612185045/http://www.nasa.gov:80/missions/",
  "daysFromTarget": 12,
  "contentTitle": "NASA Missions A-Z | NASA",
  "contentFormat": "markdown",
  "charged": true
}

A URL that was never archived gets a free row (errorType: "not_found") instead of a fake answer.

This pairs naturally with a link checker: run Broken Link Checker on your site, feed its dead links into closest mode, and you have a replacement-link sheet for your CMS.

Job 3: every version of a page, with the text

Mode snapshots lists the archived versions of one page. With the default collapse: "changes", you get one row per content change, not one per crawl, which removes most of the noise:

{ "urls": ["stripe.com/pricing"], "mode": "snapshots", "collapse": "changes", "includeContent": true, "maxContentPages": 20 }

{
  "type": "snapshot",
  "input": "stripe.com/pricing",
  "index": 2,
  "capturedAt": "2020-01-04T16:56:45Z",
  "statusCode": 200,
  "digest": "GMSUHBJQMSMCD7CWRFAY4X2OPZGY5BJW",
  "lengthBytes": 69301,
  "snapshotUrl": "https://web.archive.org/web/20200104165645/https://stripe.com/pricing",
  "rawUrl": "https://web.archive.org/web/20200104165645id_/https://stripe.com/pricing",
  "contentChanged": true,
  "daysSincePrevious": 243
}

digest is the capture's content fingerprint (same digest = same content), and rawUrl is the file exactly as captured, without the Wayback toolbar. With includeContent, each version comes as Markdown, ready to diff or to hand to an LLM: "how did this pricing page change between 2019 and 2022?"

Job 4: is this expired domain what it claims?

Mode history gives one summary row per domain. This is pets.com, a real result:

{
  "type": "history",
  "input": "pets.com",
  "firstCapturedAt": "1996-12-21T00:37:03Z",
  "lastCapturedAt": "2026-09-20T01:06:47Z",
  "lastOkMonth": "2016-04",
  "archiveAgeYears": 29.7,
  "totalCaptures": 2391,
  "monthsWithCaptures": 268,
  "monthsWithOkPages": 40,
  "monthsRedirectOnly": 228,
  "longestGapMonths": 19,
  "distinctVersions": 117,
  "redirectShare": 0.494,
  "calendarUrl": "https://web.archive.org/web/*/pets.com"
}

"Archived for almost 30 years" sounds impressive. But the last month with a real page (HTTP 200) was April 2016, and 228 of the 268 months with captures were redirect-only. If you were evaluating this as an aged domain for SEO, that is the line that matters, and it is exactly the line a marketplace listing will not show you.

What it costs

Event	Free / Starter	Scale	Business and higher
Archived URL or snapshot (one row)	$0.001	$0.0009	$0.0008
Site history summary (one per domain)	$0.003	$0.0027	$0.0024
Archived page content (optional)	$0.003	$0.0027	$0.0024

No start fee. So 1,000 archived URLs of a site cost $1, and a history check of 100 domains costs $0.30. Inputs that were never archived, invalid lines, duplicates, filtered-out captures and content that could not be downloaded are free.

A real example from my tests: two pages, closest snapshot to June 2015 with their Markdown, plus one never-archived domain = 2 captures + 2 page contents = $0.008.

Limits worth knowing

The Wayback Machine is a shared free service. The Actor defaults to one index query every 2.5 seconds and one page download every 5 seconds, and backs off when the archive is busy. Big jobs take time; that is the price of not hammering it.
Only what was archived. Pages that were never crawled, or were excluded or removed at the owner's request, are not available.
JavaScript-heavy pages may have little text in their archived HTML.
For live change alerts on a page today, use a change monitor; this tool is for the past.

Terms and responsible use

The Actor reads only the Internet Archive's public endpoints (the CDX index, the availability API and snapshot playback), at a gentle rate, without logging in. Archived pages remain their owners' work, so check copyright before republishing them. I am not affiliated with the Internet Archive; if you rely on it, consider donating.

Actor: https://apify.com/tidytools/wayback-machine-scraper

Disclosure: I built this Actor and earn from its usage on Apify. The sites above are public examples from my tests in September 2026 and have no connection to me.

Bulk WHOIS with RDAP: domain age, expiry and SPF/DMARC grade for 1,000 domains

Tidy Tools — Mon, 12 Oct 2026 02:03:44 +0000

If you manage more than a handful of domains, sooner or later one of these happens: a domain nobody remembered renewing expires, a client's domain loses its transfer lock, or someone asks "which of our 300 domains still have DMARC at p=none?"

Each of those has an answer in public data. The registry knows the expiry date and status codes. DNS knows the mail servers, SPF and DMARC policy. The problem is doing it for a list, not one domain at a time in a web form that rate-limits you after five lookups.

This post shows how to do a bulk lookup properly with RDAP (the JSON successor of WHOIS), what each registry actually publishes, and how to turn the result into an expiry and e-mail-security report.

WHOIS is being replaced by RDAP. That is good news.

Classic WHOIS is plain text over port 43, with a different format per registry. RDAP is the official replacement: HTTPS, JSON, standard field names. IANA publishes a bootstrap file that tells you which RDAP server is responsible for each TLD.

So a bulk lookup can be: find the registry's RDAP server from the IANA bootstrap (with rdap.org as a fallback), ask it, and fall back to classic WHOIS only for the country-code TLDs that have no RDAP yet (.jp, .io, .co, .it, .eu, .ru, .cn…). For .com and .net, the registry only keeps basic data, so one more request to the registrar's RDAP server gets the registrant organization and country when they are public.

That is exactly what Bulk WHOIS Lookup & DNS Lookup - Domain Age & Expiry does, together with DNS over HTTPS for the records.

One row per domain

Input can be domains, URLs or even e-mail addresses. Everything is reduced to the registrable domain and de-duplicated:

{ "domains": ["github.com", "www.github.com", "https://www.bbc.co.uk/news", "jane@shop.example.de"], "checkAvailability": true }

github.com and www.github.com become one row (and are charged once). A real result from September 2026, with TXT records shortened:

{
  "domain": "github.com",
  "inputs": ["github.com", "www.github.com"],
  "registered": true,
  "registrar": "MarkMonitor Inc.",
  "createdAt": "2007-10-09T18:20:50Z",
  "expiresAt": "2028-10-09T18:20:50Z",
  "daysToExpiry": 741,
  "domainAgeDays": 6930,
  "status": ["clientDeleteProhibited", "clientTransferProhibited", "clientUpdateProhibited"],
  "dnssec": false,
  "registrantOrg": "GitHub, Inc.",
  "registrantCountry": "US",
  "registrationSource": "rdap",
  "registrationServer": "rdap.verisign.com",
  "dns": {
    "a": ["140.82.112.4"],
    "mx": [{ "priority": 0, "host": "github-com.mail.protection.outlook.com" }],
    "caa": [{ "flags": 0, "tag": "issue", "value": "digicert.com" }, { "flags": 0, "tag": "issue", "value": "letsencrypt.org" }],
    "dmarc": "v=DMARC1; p=quarantine; sp=reject; pct=100; rua=mailto:dmarc@github.com; ruf=mailto:dmarc@github.com; fo=1"
  },
  "emailSecurity": {
    "grade": "B",
    "spfPolicy": "softfail",
    "dmarcPolicy": "quarantine",
    "spf": { "policy": "softfail", "includes": ["spf.protection.outlook.com", "_netblocks.google.com", "..."], "dnsLookups": 8, "multipleRecords": false },
    "issues": [],
    "note": "DKIM is not checked (it needs the sender's selector)."
  }
}

In one row: about 19 years old, expires in 741 days, transfer-locked, mail on Microsoft 365, SPF at 8 of the 10 allowed DNS lookups, DMARC at quarantine. That dnsLookups: 8 is the kind of thing that silently breaks SPF when someone adds one more include:.

The e-mail security grade

Every domain gets an A–F grade from its SPF and DMARC records:

A: SPF (-all or ~all) and DMARC p=reject
B: DMARC p=quarantine (or reject for only part of the mail, or testing mode)
C: DMARC p=none
D: only SPF or only DMARC, or a weak SPF (?all, +all)
F: neither

SPF include: and redirect= are followed the way a mail server does it, up to the 10-lookup limit. Every grade below A comes with at least one entry in issues (no SPF, two SPF records, p=none, too many lookups…). Domains with no MX that publish v=spf1 -all and DMARC reject also get an A: they are locked against spoofing, which is exactly what a parked domain should do.

Use case 1: an expiry report for a domain portfolio

Run your list weekly with an Apify schedule, then filter:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
domains = [line.strip() for line in open("domains.txt") if line.strip()]

run = client.actor("tidytools/bulk-domain-whois-dns-lookup").call(run_input={"domains": domains})

for d in client.dataset(run["defaultDatasetId"]).iterate_items():
    if not d.get("success"):
        continue
    days = d.get("daysToExpiry")
    unlocked = "clientTransferProhibited" not in (d.get("status") or [])
    if (days is not None and days < 60) or unlocked or d.get("emailGrade") in ("D", "F"):
        print(d["domain"], days, "unlocked" if unlocked else "", d.get("emailGrade"))

Or call it from anything that speaks HTTP:

curl -X POST "https://api.apify.com/v2/acts/tidytools~bulk-domain-whois-dns-lookup/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["github.com","bbc.co.uk"],"checkAvailability":true}'

Use case 2: an e-mail deliverability audit for clients

Agencies and MSPs can paste every client domain and sort by emailGrade. The SUMMARY record counts domains per grade, so "12 of our 80 clients have no DMARC" is one number you can put in a report. It also accepts another run's dataset (datasetId + datasetField), so the domains from a lead or contact scraper can be checked without exporting anything.

Use case 3: a first pass on domain name ideas

With checkAvailability on, a name nobody has registered looks like this:

{ "domain": "this-domain-surely-not-registered-xyz123.com", "registered": false, "availability": "likely_available",
  "availabilityNote": "Guess: the registry has no record of this domain. Premium, reserved or blocked names can still be unavailable; confirm with a registrar.",
  "registrationSource": "rdap", "dnsResolves": false }

It is deliberately worded as a guess. When the registry could not be reached, the answer is unknown, never likely_available.

What registries actually publish

This surprised me most while building it. "WHOIS data" is not one thing; it depends on the registry. From a test of 20 domains in September 2026:

TLD	Source	What came back
.com, .net, .org, .dev, .app, .fr	RDAP	Full: registrar, dates, status, name servers; registrant org/country when public
.co.uk	RDAP	Registrar, dates, status, name servers
.de (DENIC)	RDAP	Status and name servers only: no dates, registrar or holder
.com.au, .ch	RDAP	Registrar, status, name servers; no dates
.io, .co	WHOIS	Full, including registrant org and country when public
.jp	WHOIS	Dates, registrant, name servers; no registrar
.eu	WHOIS	Registrar and name servers only
.es	none	No RDAP, and WHOIS only answers pre-registered IPs: no registration data (DNS still works)

So if your portfolio is mostly .de, an "expiry report" from public data is not possible for those domains, whichever tool you use. When registration data cannot be read, the row still has the DNS records and a registrationError.

Two more honest limits: most registrant names are redacted under GDPR (values like "REDACTED" come back as null), and DKIM is not checked because it needs the sender's selector.

What it costs

$2.00 per 1,000 domains on the Free and Starter plans ($1.80 on Scale, $1.60 on Business and higher). No start fee. 10,000 domains cost $20.

A domain is charged when registration data or DNS records came back (a registry saying "not registered" is a valid answer and counts). Lookups where both RDAP/WHOIS and DNS failed, invalid lines and duplicates are free. The run logs its worst case before starting and stops cleanly at your maximum charge per run.

Actor: https://apify.com/tidytools/bulk-domain-whois-dns-lookup

Disclosure: I built this Actor and earn from its usage on Apify. github.com and the other domains are public examples from my tests in September 2026 and have no connection to me.

Content Checker alternative on Apify: 100 pages a day for $6/month

Tidy Tools — Sat, 10 Oct 2026 14:04:58 +0000

If you run a page-watching job on Apify, there is a good chance it is the Content Checker Actor: a URL, a CSS selector, a screenshot selector, an e-mail address, and a schedule that runs it every hour or day. It is simple and it works, and thousands of people set it up that way.

I needed the same thing for a lot more pages: competitor pricing pages, a few terms-of-service pages, and release notes. Running one Content Checker task per page got messy fast, and I wanted Slack alerts and less noise from dates and "5 minutes ago" counters. So I built a change monitor that accepts the same input and adds what I was missing.

Migrate in one minute

Take your existing Content Checker input and paste it into Website Change Monitor unchanged:

{
  "url": "https://www.apify.com/change-log",
  "contentSelector": "article",
  "screenshotSelector": "article",
  "sendNotificationTo": "you@your-apify-account-email.com",
  "informOnError": "true"
}

url, contentSelector, screenshotSelector, sendNotificationTo and informOnError all work as before. Keep your schedule, point it at this Actor, and you are done. The first run saves a baseline; every run after that compares with the previous one.

Then watch many pages in one run

The part that made me build it: one run can watch up to 5,000 pages, each with its own selector and name.

{
  "urls": [
    { "url": "https://www.apify.com/change-log", "userData": { "selector": "article", "name": "Apify changelog" } },
    { "url": "https://example.com/pricing", "userData": { "selector": "#plans", "name": "Competitor pricing" } }
  ],
  "monitorName": "my-watchlist",
  "ignoreCommonNoise": true,
  "watchKeywords": ["price", "out of stock"],
  "screenshotOnChange": true,
  "alertOnError": true,
  "webhookUrl": "https://hooks.slack.com/services/...",
  "notifyEmail": "you@your-apify-account-email.com"
}

ignoreCommonNoise ignores dates, times, "5 minutes ago" and view counters, so a page does not "change" on every visit.
watchKeywords alerts only when a changed line mentions one of your words.
screenshotOnChange saves a screenshot after each alerted change and links the previous one, so you get a before/after pair.
A Slack (or Discord) incoming-webhook URL gets a readable message; any other URL gets JSON, optionally signed with HMAC-SHA256.

What a change looks like

A real result (September 2026; httpbin.org/uuid returns a new ID on every request, which makes it a handy test page):

{
  "url": "https://httpbin.org/uuid",
  "status": "changed",
  "addedCount": 1,
  "removedCount": 1,
  "added": ["\"uuid\": \"762816a2-8e92-4d9a-aa9a-05b68c05601e\""],
  "removed": ["\"uuid\": \"e83ca153-a8dc-498e-b49c-80ddb4cfe18e\""],
  "significant": true,
  "screenshotUrl": "https://api.apify.com/v2/key-value-stores/.../records/shot-...-20260929034729491.jpg",
  "previousScreenshotUrl": "https://api.apify.com/v2/key-value-stores/.../records/shot-...-20260929034644443.jpg"
}

On a pricing page, number changes are pulled out for you: "numericChanges": [{ "before": "$19", "after": "$29", "context": "Pro plan $29/month" }].

A few things that kept my alerts honest:

Changing the selector does not fire a giant false alert. It saves a new baseline (reason: "settings-changed") instead.
A page that starts returning 404 or 410 is reported once as removed, and as restored when it comes back.
A page that keeps failing is reported after N failed runs in a row (informOnError: "true" means after the first), so a monitor never dies silently.
Lines that only moved around are not reported as changes.

Content Checker vs. this Actor

	Content Checker	Website Change Monitor
Your existing input	✓	✓ (same fields)
Pages per run	1	up to 5,000, per-page selector and name
Alerts	e-mail	e-mail, Slack, Discord, JSON webhook (HMAC)
Screenshots	✓ (`screenshotSelector`)	✓ before and after, linked in alerts
Ignore dates and counters	–	✓
Keyword triggers, minimum change size	–	✓
Price / number changes extracted	–	✓
AI summary of the change	–	optional, $0.005 each
Removed pages (404/410) reported	–	✓
Actor fee	none (you pay platform usage for its browser runs)	$2 per 1,000 checks, failed checks free

(Checked October 2026. E-mail in both uses Apify's send-mail Actor, which delivers to your Apify account's e-mail address; for anyone else, use Slack or a webhook to Zapier, Make or n8n.)

What it costs

$0.002 per page check, no start fee and no fee per detected change. Screenshots ($0.003 each) and AI summaries ($0.005 each) only if you turn them on. Pages that could not be read are not charged.

Monitor	Checks per month	Cost per month
1 page, every hour	720	$1.44
20 pages, once a day	600	$1.20
100 pages, once a day	3,000	$6.00

Higher Apify plans get a volume discount on the check price.

Set it up in three steps

Paste the URLs (or your Content Checker input) and give the monitor a name.
Run it once to save the baseline. The status message tells you so: "Baseline saved for N page(s). Next: create a schedule…"
In Apify Console go to Schedules → Create, pick the Actor and your input, and choose e.g. every day at 9:00.

From then on, each scheduled run reports only what changed.

Actor: https://apify.com/tidytools/website-change-monitor

Disclosure: I built this Actor and earn from its usage on Apify. Content Checker is a separate Actor by another author; the comparison above reflects its public Store page and input fields as of October 2026.

Export a trade show exhibitor list to CSV (FABTECH example), then enrich it

Tidy Tools — Fri, 09 Oct 2026 05:23:47 +0000

Two weeks before a trade show, someone on the sales team always asks the same thing: "Can we get the exhibitor list in a spreadsheet?"

The data is right there on the show's website, in a searchable directory with booth numbers, categories and links. It is just not exportable. People end up copying names by hand, or buying a list that turns out to be last year's.

Most exhibitor directories are not custom-built. A large share of shows run on one of a handful of platforms, and two of them load their data in a way that is easy to read cleanly: a2z / Personify (FABTECH, KBIS, CEDIA, HD Expo, RAPID + TCT…) and ExpoFP floor plans (SLAS, POWERGEN, Wine Paris…). This post shows how to turn either into a CSV, and then add company data to every row.

Step 1: find the directory URL

For the shows I checked on 1 October 2026, these work as they are:

Trade show	Platform	Exhibitor list URL	Exhibitors
FABTECH 2026	a2z	`https://s36.a2zinc.net/clients/SME/FABTECH2026/Public/Exhibitors.aspx?Index=All`	1,452
KBIS 2026	a2z	`https://kbis.a2zinc.net/kbis2026/Public/Exhibitors.aspx?Index=All`	609
HD Expo 2026	a2z	`https://hd.a2zinc.net/HDExpo2026/Public/Exhibitors.aspx?Index=All`	556
CEDIA Expo 2026	a2z	`https://cedia.a2zinc.net/CEDIA2026/Public/Exhibitors.aspx?Index=All`	327
Wine Paris 2026	ExpoFP	`https://wineparis2026.expofp.com/`	6,390
POWERGEN 2026	ExpoFP	`https://powergen2026.expofp.com/`	555
SLAS 2026	ExpoFP	`https://slas2026.expofp.com/`	423

For any other show, you do not need to dig: paste the event's own website, and the Trade Show Exhibitors Scraper follows the organiser's link or embed to the a2z or ExpoFP directory. A directory on some other site (an organiser's own "Exhibitors" page, for example) can be read with AI extraction into the same columns; company names are checked against the page text so the model cannot invent exhibitors.

Not supported, on purpose: Map Your Show, Swapcard and Cvent. Their terms or bot protection do not allow automated collection, so the Actor does not read them at all, and you get a free row saying so. That rules out several big US shows (SEMA, CES and CONEXPO-CON/AGG run on Map Your Show); I would rather say that up front.

Step 2: export the list

{
  "urls": [
    "https://slas2026.expofp.com/",
    "https://kbis.a2zinc.net/kbis2026/Public/Exhibitors.aspx?Index=All",
    "https://s4.goeshow.com/ime/annual/2026/exhibitor_list.cfm"
  ],
  "countries": ["Italy", "Germany"],
  "maxExhibitorsPerDirectory": 200
}

Every event comes back in the same schema, so a multi-show list is one table. A real a2z row (FABTECH 2026, profile read):

{
  "platform": "a2z (Personify)",
  "eventName": "FABTECH 2026",
  "name": "1960 SERAVESI",
  "booth": "C8119",
  "categories": ["Bending/Forming", "Flanging Machines", "Plate & Structural Fabricating", "Angle Bending Rolls"],
  "website": "https://www.1960seravesi.com/",
  "country": "Italy",
  "city": "Calcinato",
  "postalCode": "25011",
  "address": "Via Statale 10T",
  "companyPhone": "+39030-6091301",
  "profileUrl": "https://s36.a2zinc.net/clients/SME/FABTECH2026/Public/eBooth.aspx?BoothID=539883&EventID=128",
  "chargedEvent": "exhibitor"
}

And an ExpoFP row (SLAS 2026), which comes with the event dates:

{
  "platform": "ExpoFP",
  "eventName": "SLAS 2026",
  "eventStartDate": "2026-02-07",
  "eventEndDate": "2026-02-11",
  "name": "10x Genomics",
  "booth": "849",
  "categories": ["Genomics", "Fluorescence Assays", "Drug Development", "Reagents"],
  "website": "http://www.10xgenomics.com/",
  "country": "United States",
  "city": "Pleasanton",
  "companyEmail": "support@10xgenomics.com",
  "companyPhone": "(925) 401-7300"
}

Filters (keywords over name, description and categories; category; country, where "US", "USA" and "United States" are the same) are applied before charging. Export as CSV, Excel or JSON from the dataset.

What two real runs cost

From my test runs on 30 September 2026, at Free plan prices:

Run	Directories	Rows	Cost
a2z + ExpoFP, max 5 per directory	semashow.com (links Map Your Show: free `unsupported` row), FABTECH 2026, SLAS 2026	10 with details + 1 free row	$0.03 (7.1 s)
a2z + AI, max 25 per directory	FABTECH 2026, HD Expo 2026, IMX 2026 and NACUBO 2026 (AI), one Swapcard URL (free row)	50 with details + 50 AI	$0.30 (43 s)

Costs in the table are calculated at the current price list (the two runs were done at the earlier $2 per 1,000 exhibitors and cost $0.02 and $0.25). A whole directory costs about what its size says: FABTECH 2026 with 1,451 exhibitor profiles is about $4.35.

Step 3: add company data to every row

An exhibitor list tells you who is there. Before you plan meetings, you usually want to know what each company does and how to reach it. The website column feeds straight into Google Maps Lead Enrichment (it takes any list of company websites, not only Google Maps leads), which reads each company's own site and returns an industry label, a one-line description, the contact channels the site publishes (company e-mail, phone, social profiles) and its tech stack.

Pass the exhibitor run's dataset and keep the booth next to each result:

{
  "datasetId": "<dataset ID of the exhibitor run>",
  "datasetField": "website",
  "datasetKeepFields": ["name", "booth", "eventName"]
}

Or the whole pipeline from Python with apify-client:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

show = client.actor("tidytools/trade-show-exhibitor-list-scraper").call(run_input={
    "urls": ["https://s36.a2zinc.net/clients/SME/FABTECH2026/Public/Exhibitors.aspx?Index=All"],
    "countries": ["Germany", "Italy"],
})

companies = client.actor("tidytools/company-website-enrichment").call(run_input={
    "datasetId": show["defaultDatasetId"],
    "datasetField": "website",
    "datasetKeepFields": ["name", "booth", "eventName"],
})

for row in client.dataset(companies["defaultDatasetId"]).iterate_items():
    kept = row.get("source") or {}          # the exhibitor fields you asked to keep
    print(kept.get("booth"), kept.get("name"), row.get("industry"), row.get("description"))

Enrichment is $6 per 1,000 enriched companies (AI description included); unreachable websites are free.

What it costs

Row type	Free / Starter	Scale	Business and higher
Exhibitor with details (website, description, location or categories)	$3.00 / 1,000	$2.70	$2.40
Name and booth only	$0.50 / 1,000	$0.45	$0.40
Exhibitor read with AI (other sites)	$3.00 / 1,000	$2.70	$2.40

No start fee. Filtered-out exhibitors, directories that cannot be read, unsupported platforms and invalid lines are free. Each row's chargedEvent says which price applied. For a quick name-and-booth list, turn off Read exhibitor profiles on a2z shows.

Limits worth knowing

a2z profiles take one extra request each (three at a time, with pauses), so a 1,500-exhibitor show takes a few minutes. ExpoFP is a single request.
AI extraction reads one page. Paginated lists need one URL per page.
Fewer rows than the show advertises? Marketing numbers ("1,000+ exhibitors") often count co-exhibitors the directory does not list. The run's SUMMARY compares the directory's own total with what you got, per directory.

Responsible use

This collects company-level information that organisers publish so visitors can find exhibitors. No contact persons, staff names, personal e-mails or personal phone numbers are extracted; a company e-mail is kept only when it is a generic mailbox (info@, sales@, support@). robots.txt is respected on every request, and platforms that do not allow automated collection are skipped. Check the event's terms, and follow GDPR, CAN-SPAM and CASL before you contact anyone.

Actor: https://apify.com/tidytools/trade-show-exhibitor-list-scraper

Disclosure: I built both Actors and earn from their usage on Apify. The shows and companies above are public examples from my test runs of 30 September and 1 October 2026 and have no connection to me.

Clean article text + a TL;DR from any news URL in one API call

Tidy Tools — Thu, 08 Oct 2026 02:07:11 +0000

If you have ever tried to feed news articles to an LLM, you know the boring part is not the summary. It is everything before it: cookie banners, "related stories", share buttons, newsletter boxes, author bios, and pages that are not articles at all (a home page, a section front, a login wall).

I built a small API that does the boring part and then, only if you ask, the summary. Below are real calls I made on 1 October 2026, with the actual responses.

Step 1: just the article (no AI)

curl --request POST \
  --url 'https://article-extractor-ai-summarizer-api.p.rapidapi.com/api/v1/article' \
  --header 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  --header 'X-RapidAPI-Host: article-extractor-ai-summarizer-api.p.rapidapi.com' \
  --header 'Content-Type: application/json' \
  --data '{"url": "https://www.theguardian.com/technology/2026/sep/29/oura-ring-public-offering", "maxChars": 400}'

The response came back in 0.48 s (plain HTTP, no browser needed). Trimmed to the fields that matter:

{
  "success": true,
  "isArticle": true,
  "pageType": "article",
  "articleType": "NewsArticle",
  "title": "Smart ring maker Oura puts off initial public offering due to market ‘uncertainty’",
  "authors": ["Guardian staff reporter", "Associated Press"],
  "publishedAt": "2026-09-29T17:57:01.000Z",
  "modifiedAt": "2026-09-29T18:20:35.000Z",
  "publisher": "The Guardian",
  "language": "en",
  "tags": ["Technology", "IPOs", "AI (artificial intelligence)", "Business", "US news"],
  "paywalled": false,
  "wordCount": 296,
  "readingTimeMinutes": 1,
  "tokensEstimate": 445,
  "format": "markdown",
  "markdown": "…(body as Markdown, cut at 400 characters because of maxChars)…",
  "truncated": true,
  "summaryGenerated": false
}

A few details I cared about:

authors is an array, not a joined string, and the dates are ISO 8601, so you can filter by freshness without parsing.
wordCount and tokensEstimate describe the whole article even when maxChars cuts the body. Handy for deciding whether a page fits your context window before you download all of it.
format can be markdown, text or both.

Step 2: the same page, with a TL;DR

The summary lives on a separate endpoint, /api/v1/article/summary, so extraction-only calls never touch the model (and never use your summary quota).

curl --request POST \
  --url 'https://article-extractor-ai-summarizer-api.p.rapidapi.com/api/v1/article/summary' \
  --header 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  --header 'X-RapidAPI-Host: article-extractor-ai-summarizer-api.p.rapidapi.com' \
  --header 'Content-Type: application/json' \
  --data '{"url": "https://www.theguardian.com/technology/2026/sep/29/oura-ring-public-offering", "summary": "tldr"}'

1.74 s, and the summary block looked like this:

"summary": {
  "style": "tldr",
  "language": "same as source",
  "maxWords": 30,
  "text": "Oura postpones its initial public offering due to market uncertainty despite strong demand and revenue growth.",
  "inputChars": 1866,
  "inputTruncated": false,
  "tokens": { "input": 575, "output": 20 }
}

You get the token counts back, so you can see what the model actually read.

Bullets, in another language

Same article, "summary": "bullets", "maxWords": 80, "language": "Spanish" — 3.92 s, 7 bullets. The first three:

- Oura Inc está retrasando su oferta pública inicial debido a la incertidumbre del mercado.
- La empresa había planeado vender 50m acciones en la oferta pública inicial a un precio entre $40 y $44.
- Casi tres cuartas partes de las acciones serían vendidas por accionistas actuales.

There is also "summary": "paragraph" and a focus option (for example "focus": "numbers and dates"). If you already have the text, send text instead of url.

Step 3: what happens with a page that is not an article

This is the part I wanted most. Summarising a home page gives you a confident paragraph about nothing. So the API checks the page type first and refuses before any AI runs:

--data '{"url": "https://www.bbc.com/news"}'

{
  "success": false,
  "error": {
    "code": "not_an_article",
    "message": "This page is not an article: this page lists articles (a section, category or tag page) rather than being one.",
    "details": { "pageType": "listing", "finalUrl": "https://www.bbc.com/news" }
  }
}

That came back in 0.67 s with HTTP 422. If you do want the page anyway, pass "allowNonArticle": true and you will get it with isArticle: false.

Section fronts were the hard case. My first test run let the Guardian's technology section through as an "article" (no author, no date). The check now also looks at the page's structured data (a CollectionPage is not an article, even when og:type says so), at how much of the page is headline cards with short teasers, and at URL shapes like /section/, /category/ and /tag/. In a re-run, 15 section, category and tag pages (Guardian, BBC, Reuters, Spiegel, TechCrunch, The Verge, Medium, Cloudflare's blog and others) all came back 422 with pageType: listing, and 13 real articles (news in English, German and Japanese, Wikipedia in English and Japanese, a Paul Graham essay, Medium, Substack, Cloudflare and personal blog posts) were all still accepted. It is still a heuristic, so if you feed it mixed URLs, pageType and publishedAt are worth a glance.

And some sites simply refuse automated requests. One news site I tried answered the first extraction and then returned HTTP 405 to the next calls; the API reported that as 422 target_blocked instead of making something up.

Python: summarise a list of links

import requests

HOST = "article-extractor-ai-summarizer-api.p.rapidapi.com"
HEADERS = {"X-RapidAPI-Key": "YOUR_RAPIDAPI_KEY", "X-RapidAPI-Host": HOST}

def tldr(url: str) -> str | None:
    r = requests.post(f"https://{HOST}/api/v1/article/summary",
                      json={"url": url, "summary": "tldr"}, headers=HEADERS, timeout=60)
    body = r.json()
    if not body.get("success"):
        print(url, "->", body["error"]["code"])   # not_an_article, target_blocked, timeout...
        return None
    return body["summary"]["text"]

for link in ["https://www.theguardian.com/technology/2026/sep/29/oura-ring-public-offering",
             "https://www.bbc.com/news"]:
    print(link, "=>", tldr(link))

Limits

Web pages only (for PDFs and Word files use a document converter).
The model reads up to about 6,000 tokens of the article; longer articles are summarised from the start, and inputTruncated tells you.
At most one real browser render per call, for sites that need JavaScript; most news pages do not.
30 s timeout.

Price

On RapidAPI there is a free plan (200 requests and 30 summaries a month, hard limit; RapidAPI may ask for a card even for free plans). The plans as of 1 October 2026:

Plan	Per month	Extraction requests	Summaries
Basic	$0	200	30
Pro	$12.99	10,000	1,500
Ultra	$39.99	50,000	6,000
Mega	$99.99	150,000	15,000

Every call counts as a request; only /article/summary calls count as summaries. RapidAPI also has its own bandwidth fee above 10 GB a month, which is theirs, not mine.

Try it: https://rapidapi.com/tidytools/api/article-extractor-ai-summarizer-api

If you would rather run it in bulk without code, the same summariser is an Apify Actor, Article Summarizer & Web Page Summarizer - AI Text Summarizer ($6 per 1,000 pages).

Disclosure: I built this API and the Apify Actor, and I earn money when people use them. All outputs above are real responses from 1 October 2026; the article quoted is Associated Press/Guardian reporting, used here only as a test input.

Track Shopify competitor prices daily with products.json (no API key)

Tidy Tools — Wed, 07 Oct 2026 02:02:27 +0000

If you sell on Shopify, your competitors probably do too. And every Shopify storefront has a public, documented-by-accident API that most people never look at:

https://www.allbirds.com/products.json?limit=250&page=1

Open that in a browser. It is the store's entire catalog as JSON: every product, every variant, SKUs, prices, compare-at prices and whether each size is in stock. No login, no API key, no HTML parsing. Shopify serves it to every visitor (unless the store turns it off).

That makes Shopify price monitoring much simpler than scraping product pages. This post shows how to turn that feed into a clean export and a daily price-drop alert, and where its limits are.

What the feed gives you (and what it does not)

Data	Endpoint	Limit
Products and variants	`/products.json?limit=250&page=N`	250 per request; at most 100 pages, so 25,000 products per catalog or collection
One collection	`/collections/<handle>/products.json`	same
Collections	`/collections.json`	250 per request
Store facts	`/meta.json`	name, myshopify domain, default currency, country, product count

Three things it does not give you:

Currency. products.json has prices but no currency field. You have to read it from the store page or meta.json, and stores using Shopify Markets convert prices by visitor country.
Stock quantities. You get available: true/false per variant, never "3 left".
Anything on headless storefronts. Stores built on Hydrogen (fashionnova.com in my tests) answer 404.

One clean row per product

I wrapped all of this in an Apify Actor, Shopify Product Scraper & Shopify Price Monitor, so you can paste store URLs, domains, collection URLs or product URLs and get a flat table. It also checks robots.txt first and waits one second between requests per store by default.

{
  "stores": ["https://www.allbirds.com", "gymshark.com", "https://kith.com/collections/mens-footwear"],
  "maxProductsPerStore": 300,
  "onlyOnSale": true
}

A real row (30 September 2026), shortened:

{
  "storeName": "Allbirds",
  "title": "Women's Allbirds Flip Flop - Dusty Pink",
  "url": "https://www.allbirds.com/products/womens-allbirds-flip-flop-dusty-pink",
  "vendor": "Allbirds",
  "productType": "Shoes",
  "price": 25,
  "priceMax": 25,
  "compareAtPrice": 50,
  "onSale": true,
  "discountPercent": 50,
  "currency": "USD",
  "currencySource": "store page",
  "available": true,
  "variantsCount": 7,
  "variantsAvailable": 1,
  "skus": ["A12513W050", "A12513W060", "A12513W070", "A12513W080", "A12513W090", "A12513W100", "A12513W110"],
  "options": [{ "name": "Size", "values": ["5", "6", "7", "8", "9", "10", "11"] }],
  "publishedAt": "2026-09-25T23:58:13.000Z",
  "charged": true
}

That single row already answers a competitive question: a new product, published five days earlier, at half price, with only one of seven sizes left. discountPercent and variantsAvailable are computed for you; the raw variants array (SKU, price, compare-at, availability, weight per size) is in the row too. Set rowPer to variant if you want one spreadsheet line per size or colour; you still pay once per product.

Filters run before charging: keywords, excluded words, vendors, product types, tags, price range, in stock only, on sale only, published since a date, or only certain collections.

The real test run

Seven inputs, 300 products per store: allbirds.com, gymshark.com, a Kith collection, colourpop.com, one Tentree product URL, fashionnova.com (headless, no feed) and example.com (not Shopify).

Result: 1,201 products from 5 stores in 12.6 seconds, plus 2 free error rows. Cost on the Free plan: 1,201 × $0.001 = $1.20.

The two failures come back as rows that explain themselves, and are not charged:

{ "input": "https://www.fashionnova.com", "success": false, "errorType": "not_found",
  "error": "products.json could not be read: HTTP 404 (not found) (the store has turned off its public product feed, or it is a headless storefront) (not charged)", "charged": false }

Turn it into a daily price monitor

A one-off export is useful once. Price monitoring needs a baseline and a diff. Turn on onlyChanges and give the watchlist a name:

{ "stores": ["https://www.tentree.com", "https://colourpop.com"], "onlyChanges": true, "monitorName": "competitors" }

Run it once. This first run returns the products and saves them as the baseline.
In Apify Console go to Schedules → Create, pick this Actor and input, and run it daily.
Each later run returns only new and changed products, with what changed.

A changed row looks like this (this is the shape from the unit tests; in my real test, a second Tentree run two minutes later found 0 changes and cost nothing):

{
  "title": "Women's Allbirds Flip Flop - Dusty Pink",
  "price": 20,
  "changeType": "changed",
  "changes": ["price_drop", "back_in_stock"],
  "previousPrice": 25,
  "variantChanges": [
    { "title": "5", "change": "price,available", "price": 20, "previousPrice": 25, "available": true, "previousAvailable": false }
  ]
}

Change types: price_drop, price_increase, back_in_stock, out_of_stock, compare_at_price, variant_added, variant_removed. Products that disappear get a free "removed" row (only when the whole catalog was read with no filters, so "not read this time" is never reported as "removed"). Connect the Slack, email or webhook integration on the Actor and the changes arrive as alerts.

Two guards that kept my test alerts honest:

Unchanged products are free. A daily check on a quiet store costs nothing.
Mass changes are held back. If more than half of a store's catalog (stores with at least 10 products) changes in one run, or the store moves to a different myshopify shop or domain, every row is still output but marked suspiciousChange: true, nothing is charged for that store, and the old baseline is kept until you accept the new one. A store-wide currency switch should not be billed as thousands of "price changes".

Bonus: which of these sites run on Shopify?

Set mode to detect and paste domains:

{ "stores": ["allbirds.com", "gymshark.com", "kith.com", "example.com", "wikipedia.org"], "mode": "detect" }

{ "store": "https://gymshark.com", "isShopify": true,
  "detectedBy": ["meta.json", "powered-by header", "Shopify theme script", "products.json"],
  "storeName": "Gymshark US", "myshopifyDomain": "gymsharkusa.myshopify.com",
  "currency": "USD", "country": "US", "productsCount": 10088, "collectionsCount": 526 }

Sites that are not Shopify stores are free, which makes this a cheap first filter on a lead list.

What it costs

Event	Free / Starter	Scale	Business and higher
Product (all variants included)	$1.00 / 1,000	$0.90	$0.80
Collection (collections mode)	$0.30 / 1,000	$0.27	$0.24
Shopify store detected	$1.00 / 1,000	$0.90	$0.80

No start fee. Not charged: filtered-out products, unchanged products in monitoring mode, removed-product rows, suspicious change sets, non-Shopify sites, password-protected stores and stores without a public feed. Max products per store (default 1,000) and Max products (default 10,000) cap the run, and it logs its worst case before starting.

Limits worth knowing

25,000 products per catalog or collection is Shopify's own cap. For bigger stores, read collection by collection.
Prices follow the requester's country. Runs come from the US, so Shopify Markets stores usually answer in USD. currency says what the numbers are; storeCurrency shows the store's default when it differs.
Some stores block data-center traffic or disable the feed; you get a free row saying which.

Terms and responsible use

This reads only the public storefront endpoints Shopify serves to every visitor, checks robots.txt first, and returns product data only: no customer, order or personal contact data. Product texts and images belong to the stores, so use the data for monitoring and analysis, not to republish their catalog as your own.

Actor: https://apify.com/tidytools/shopify-store-products-scraper

Disclosure: I built this Actor and earn from its usage on Apify. The stores above are public examples from my test runs of 30 September 2026 and have no connection to me.

Clean an email list without SMTP pings: typos, disposable and dead domains

Tidy Tools — Tue, 06 Oct 2026 12:32:46 +0000

Every outbound campaign starts with the same chore: a spreadsheet of addresses from a form, an event, a CRM export or a scraper, and the nagging feeling that a chunk of them will bounce.

Most "email verifier" tools answer this by connecting to the recipient's mail server and pretending to send a message (an SMTP "ping"). That used to work. Today it mostly does not: Gmail, Outlook, Yahoo and the big corporate gateways either block those probes or answer "yes" to every address, and the IP doing the probing can end up on a blocklist.

So I built a validator that deliberately skips SMTP and does everything that can be known without talking to a mail server. This post shows what that catches, what it cannot catch, and how to run it on a list or straight on a scraper's output.

What you can know without touching the mailbox

For every address, Email List Cleaner - Bulk Email Validator checks:

Syntax per RFC 5321/5322, including quoted local parts, internationalized domains and inputs like "Jane Doe <jane@example.com>" or mailto: links.
Does the domain exist, and does it accept mail? MX records over DNS-over-HTTPS, including domains that publish a null MX (an explicit "we take no mail") and the implicit fallback to A/AAAA.
Disposable domains: a community list of about 9,200 temporary-mail domains, plus disposable mail servers hidden behind custom domains.
Role, no-reply and free-provider addresses: info@, sales@, noreply@, Gmail, Outlook, GMX, QQ, Naver…
Typos: gmial.com, gmail.con, .cmo, with a didYouMean suggestion.
Mail provider from the MX hosts: Google, Microsoft, Proofpoint, Mimecast, Zoho and others.

Each address gets a verdict, a 0–100 score and a reason in plain English:

Verdict	When	What to do
`valid`	Domain exists and accepts mail, nothing suspicious (role addresses stay valid with a lower score)	Keep
`risky`	Disposable, likely typo, no-reply, or several weak signals	Fix the typo, or drop it for cold outreach
`invalid`	Bad syntax, domain does not exist, null MX, no mail server	Remove
`unknown`	DNS did not answer even after a retry	Run again later (not charged)

A real run

Input (you can also paste text, upload a CSV or point it at another run's dataset):

{
  "emails": ["jane.doe@gmail.com", "info@stripe.com", "someone@gmial.com", "test@mailinator.com"]
}

A clean row from a CSV upload (1 October 2026), with the CSV's own Name and Company columns carried along:

{
  "input": "jane@apify.com",
  "verdict": "valid",
  "score": 90,
  "reason": "domain accepts e-mail; mailbox not verified (no SMTP check)",
  "domain": "apify.com",
  "hasMx": true,
  "mailProvider": "Google",
  "disposable": false,
  "role": false,
  "freeProvider": false,
  "mxRecords": [{ "priority": 1, "host": "aspmx.l.google.com" }, { "priority": 5, "host": "alt1.aspmx.l.google.com" }],
  "catchAll": null,
  "mailboxChecked": false,
  "charged": true,
  "csvRow": 2,
  "sourceFields": { "Name": "Jane", "Company": "Acme" }
}

Note the honest bits: mailboxChecked: false and catchAll: null. Those can only be known over SMTP, so the tool says so instead of guessing.

The typo case is the one that saves real leads:

{
  "input": "someone@gmial.com",
  "verdict": "risky",
  "score": 20,
  "reason": "disposable (temporary) e-mail domain",
  "didYouMean": "someone@gmail.com",
  "reasons": [
    "disposable (temporary) e-mail domain",
    "possible typo: did you mean gmail.com?",
    "no MX record: mail would go to the domain's A/AAAA address (unusual for real mailboxes)"
  ]
}

Other results from the same test:

user@example.com → invalid: "the domain does not accept e-mail (null MX record)"
zed@gmail.con → invalid: "the domain does not exist (NXDOMAIN)", with didYouMean: "zed@gmail.com"
noreply@github.com → risky: "no-reply or system mailbox"
bad..address@gmail.com → invalid: "consecutive dots in the local part", and not charged

Besides the table, every run saves two plain-text records, VALID_EMAILS and VALID_AND_RISKY_EMAILS, one address per line, ready to paste into your sending tool.

Validate a scraped lead list in one pipeline

Where this gets useful is right after a scraper. The Email Extractor / Website Contact Details Scraper returns rows with an emails array per website. Feed its dataset straight in; arrays are expanded and you can keep the website next to each verdict:

{ "datasetId": "<dataset ID of the contact scraper run>", "datasetField": "emails", "keepFields": ["url"] }

Or from Python with the official apify-client package:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

contacts = client.actor("tidytools/website-contact-extractor").call(
    run_input={"domains": ["stumptowncoffee.com", "linear.app"]})

check = client.actor("tidytools/bulk-email-validator").call(
    run_input={"datasetId": contacts["defaultDatasetId"], "datasetField": "emails", "keepFields": ["url"]})

for row in client.dataset(check["defaultDatasetId"]).iterate_items():
    if row.get("success"):
        print(row["verdict"], row["email"], row.get("reason"))

clean = client.key_value_store(check["defaultKeyValueStoreId"]).get_record("VALID_EMAILS")
print(clean["value"])

For a Google Sheet, publish it as CSV (File → Share → Publish to web → CSV) and paste the link into the CSV field. The column named like "email" is found automatically; set csvColumn if yours is called something else.

What it costs

$0.50 per 1,000 addresses on the Free and Starter plans ($0.45 on Scale, $0.40 on Business and higher). No start fee.

You pay only for unique addresses whose domain was actually checked. Free: syntax errors, unknown results, merged duplicates, and lines without an @ (empty CSV cells, notes). Every row says charged: true or false.

So a 10,000-address list costs at most $5. The run logs its worst case before it starts (Plan: 12 e-mail addresses × $0.0005 = at most $0.006), stops cleanly at your maximum charge per run, and if Apify restarts the run, addresses already done are not charged again.

What it cannot tell you

I would rather you know this before you rely on it:

valid does not mean "this mailbox exists". A made-up name at a real company domain (nobody123@stripe.com) comes back valid. If you need mailbox-level verification, use an SMTP verifier and accept its trade-offs.
Brand-new throwaway domains may not be on the disposable list yet.
Role addresses are not thrown out. info@ and sales@ often do receive mail; filter on role if you only want people.
Free providers are not "risky". freeProvider: true is just a flag, handy for separating B2B from consumer sign-ups.

Use the verdicts to remove what is certainly bad (invalid), review what is doubtful (risky), and keep the rest.

Responsible use

Only public DNS records are queried; no email is sent and no mail server is contacted, so the people behind the addresses are never touched. Addresses are sent to the validation backend only to be checked; they are not stored there and request contents are not logged. A valid verdict is not consent: validate only lists you are allowed to process, and follow GDPR, CAN-SPAM and CASL when you send.

Actor: https://apify.com/tidytools/bulk-email-validator

Disclosure: I built this Actor and the contact scraper, and I earn from their usage on Apify. All outputs above are real results from the Actor's test run of 1 October 2026; stripe.com, github.com and the other domains are public examples with no connection to me.

I benchmarked my website-to-Markdown crawler against Apify's Website Content Crawler on 5 real sites

Tidy Tools — Mon, 05 Oct 2026 10:45:36 +0000

Disclosure first: I built one of the two tools in this test, Website Markdown Crawler. The other one is Apify's official Website Content Crawler (WCC), which about 12,000 people use every month and which has 234 reviews averaging 4.6 stars. I have a conflict of interest, so I'm publishing the method and every per-run number with this post (benchmark page). You can check the numbers and tell me where I got something wrong.

The setup

I used five public sites, each built a different way, and capped each run at 25 pages. Both crawlers got the same start URL and the same path scope.

Site	Start URL	Kind
Python tutorial	docs.python.org/3/tutorial/	Sphinx, static HTML
Docusaurus docs	docusaurus.io/docs/	Docusaurus (pre-rendered React)
Next.js docs	nextjs.org/docs/app/	Next.js, JS-heavy
Intercom Help	intercom.com/help/en/	Help center
Cloudflare Blog	blog.cloudflare.com	Blog with long posts and code

Each tool ran with the settings a new user gets:

WCC: the Console prefill (adaptive Playwright crawler, "readable text" extraction, robots.txt on, default 8 GB memory), plus maxCrawlPages: 25.
Mine: only the URL and maxPages: 25.

For each tool's cheapest mode (WCC's Cheerio crawler, my fast mode), I did one extra run on the Next.js docs.

For quality, a small script samples 5 pages per site and pulls the main content out of each page's HTML: headings, paragraphs, <pre> code blocks and tables. It then checks how many of those made it into the Markdown. It also counts template lines, meaning lines that repeat on at least half of a run's pages, such as menus, footers and "Was this helpful?".

Price and speed

Site	WCC time	WCC compute (CU per 1,000 pages)	WCC $ per 1,000 pages*	My time	My $ per 1,000 pages
Python tutorial	56 s	7.3	$1.55	4 s	$1.00
Docusaurus	84 s	6.9	$1.57	4 s	$1.00
Next.js	200 s	17.1	$3.64	4 s	$1.00
Intercom Help	183 s	15.6	$3.65	91 s**	$1.00
Cloudflare Blog	159 s	11.8	$2.77	4 s	$1.00

* WCC bills platform usage, and the rate on my account was $0.20 per compute unit. Your cost depends on your plan's compute-unit price and on the memory you choose. Lower memory lowers the cost.
** In one request, my backend stalled for about 90 s before a retry succeeded. It's a known issue, listed at the end.

In cheapest mode on the Next.js docs, WCC's Cheerio crawler took 169 s and used 10.5 CU per 1,000 pages (about $2.22). My fast mode took 4 s and costs $1.00.

Content quality (first run, before any changes)

This is the fair, blind comparison. I didn't tune anything for these sites beforehand.

Site	Tool	Headings found	Paragraphs found	Code blocks found (fenced)	Template lines per page
Python tutorial	WCC	43/43	237/242	131/131 (0 fenced)	0.0
	Mine	43/43	239/242	131/131 (0 fenced)	6.6
Docusaurus	WCC	72/92	258/273	60/62 (11 fenced)	0.0
	Mine	37/40	76/76	18/18 (18 fenced)	2.9
Next.js	WCC	7/68	0/82	0/33	6.8
	Mine	66/68	82/82	33/33 (33 fenced)	13.9
Intercom Help	WCC	82/103	106/309	–	3.3
	Mine	67/71	176/181	2/2 (0 fenced)	5.8
Cloudflare Blog	WCC	46/56	196/199	7/7 (7 fenced)	0.0
	Mine	47/52	176/179	7/7 (7 fenced)	35.3

What stood out:

WCC is much better at removing page chrome. Its Readability-based extraction left zero repeated template lines on the blog and on Docusaurus. My crawler left 35 per page on the blog: language menus, the newsletter box, social links. That was a clear loss.
Next.js with WCC defaults went wrong. For 22 of the 26 pages, the output was only the docs version picker ("Latest 16.3.7 / Version 15 …"), about 115 characters. I ran it once and didn't dig into the cause, so treat it as one data point. With WCC's Cheerio crawler, the same site came back with full text, but only 4 of 23 code blocks survived.
On Intercom Help, WCC kept the headings but dropped much of what sat under them: lists, tables and FAQ answers. That brought it to 34% of sampled paragraphs.
Code on Sphinx sites: neither tool produced fenced code blocks for the Python docs. WCC also backslash-escaped the code (\>>> x \= 1), with 2,562 escapes in 17 pages.
Heading permalinks ([¶](#…) or a zero-width link next to every heading) showed up in both tools' output: 120–184 per site.
Neither tool returned duplicate pages. WCC returned a few more pages than the cap (26–36 for a cap of 25).

What I changed after seeing this

I fixed the categories where my crawler lost or where both tools did badly:

Main-content extraction: if a page has one clear <article>, role="main" or <main> element that contains the H1 and at least 30% of the page's text, only that element is converted. If the result is almost empty, the whole page is converted instead.
Code fences for bare <pre> blocks (Sphinx/Pygments), with the language taken from highlight-python3-style wrapper classes.
Tables written without closing tags (<td>a<td>b, which HTML allows) got their implied closing tags back. Before that, Docusaurus tables were flattened into loose paragraphs.
Absolute links, so chunks still work once they're cut out of the page.
Heading permalinks, standalone widget labels ("Copy page", "Was this helpful?", "Scroll to top") and HTML entities in front matter are removed or decoded.
Crawl order: links inside page content first, then menus, then the sitemap. Other languages, old versions and author/tag listing pages come last. Before this change, 24 of my 25 Docusaurus pages were archived 2.x docs, which the sitemap lists first, and 8 of 25 blog pages were translations of one post.

Re-run with the same settings:

Site	Template lines per page (before → after)	Code blocks fenced (before → after)*	Permalinks
Python tutorial	6.6 → 0.0	0/131 → 131/131	137 → 0
Docusaurus	2.9 → 0.9	18/18 → 44/44	184 → 0 (tables 0/2 → 9/9)
Next.js	13.9 → 3.4	33/33 → 22/22	0 → 0
Intercom Help	5.8 → 0.6	0/2 → no code on sampled pages	0 → 0
Cloudflare Blog	35.3 → 8.6	7/7 → 7/7	0 → 0

* Sample pages differ between the two runs, because the new crawl order reaches different pages.

These "after" numbers were measured on the same five sites I tuned against, so they're optimistic. The honest comparison is the blind table above.

Where WCC is still the better choice

Blog-style pages: WCC still leaves less clutter. It had 0 template lines per page on the Cloudflare Blog, and mine has 8.6 (related tags, "follow us", the subscribe form).
Knobs and integrations: WCC gives you wait-for-selector, click-to-expand, scrolling, iframes, saved HTML, screenshots and files, and vector-database integrations. My crawler has far fewer options.
Track record: WCC had about 2.96 million runs in the last 30 days with a 96.6% success rate. Mine is new.
JavaScript-only sites: all five test sites serve server-rendered HTML. On a site that renders its content only in the browser, WCC's adaptive crawler may do better. My crawler does fall back to a real browser, but this benchmark didn't test that.

Known issues on my side

Occasionally one backend request stalls for 40–90 s before a retry succeeds.
In link cards (a heading inside an <a>), the title and description can run together without a space.
Blog footers inside the main element still get through.

Reproduce it

The method (sites, settings, how recall and clutter are counted) and the per-run numbers (time, cost, recall, clutter, size) are on the benchmark page, with the summary table as a CSV download.

My crawler costs $1 per 1,000 pages over HTTP and $2.50 per 1,000 when a page needs a real browser. Failed, blocked and duplicate pages are free. If you try it on your own docs and something looks wrong, tell me; the benchmark was the easy part.

What does this company use? Tech stack + email provider from a domain

Tidy Tools — Mon, 05 Oct 2026 10:45:31 +0000

"Which companies on my list use Google Workspace?" "Who sends mail through HubSpot?" "Is this prospect on Next.js or WordPress?"

Sales, partnerships and dev-tool marketing teams ask these questions all day. Paid enrichment databases answer them from a snapshot that may be months old. But most of the answer is public and live: in the website's HTML and headers, and in the domain's DNS.

I built two APIs that read exactly that. Here is what they returned for real companies on 1 October 2026, and how to read the result without fooling yourself.

Part 1: the website — `techstack`

curl --request GET \
  --url 'https://tech-stack-detector2.p.rapidapi.com/api/v1/techstack?url=posthog.com' \
  --header 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  --header 'X-RapidAPI-Host: tech-stack-detector2.p.rapidapi.com'

0.49 s:

{
  "success": true,
  "url": "https://posthog.com/",
  "renderedWith": "http",
  "count": 5,
  "technologies": [
    { "name": "Gatsby", "category": "Static site generator", "confidence": "high", "version": "4.25.9",
      "evidence": ["meta: generator: Gatsby 4.25.9", "html: <div id=\"___gatsby\">"] },
    { "name": "React", "category": "JavaScript framework", "confidence": "high", "impliedBy": "Gatsby" },
    { "name": "Tailwind CSS", "category": "UI library", "confidence": "high" },
    { "name": "Cloudinary", "category": "CDN", "confidence": "medium",
      "evidence": ["media: https://res.cloudinary.com/…"] },
    { "name": "Vercel", "category": "Hosting / PaaS", "confidence": "high",
      "evidence": ["header: x-vercel-id: …", "header: x-vercel-cache: HIT"] }
  ]
}

Every detection carries its evidence, so you can check why it was reported. For linear.app (1.41 s) it found Next.js (from an x-nextjs-cache header and _next/static scripts), React (implied), Cloudflare and Google Cloud (via: 1.1 google, medium confidence).

This is the part every "BuiltWith-style" tool does. The interesting part is the next one.

Part 2: the domain — DNS tells you the tools behind the website

A website shows you the front end. DNS shows you the company:

MX records say who actually hosts their email (Google Workspace, Microsoft 365, their own servers…).
SPF include:s list every service allowed to send mail as them — marketing automation, transactional email, support desks.
TXT verification records are left behind when someone proves domain ownership to a SaaS vendor: Slack, Notion, Atlassian, Stripe, OpenAI…

The Company Enrichment API combines this with the website, the registration record and the careers page:

curl --request GET \
  --url 'https://company-enrichment-api13.p.rapidapi.com/api/v1/company/enrich?domain=linear.app' \
  --header 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  --header 'X-RapidAPI-Host: company-enrichment-api13.p.rapidapi.com'

2.63 s, 4 pages read (home, about, contact, careers), no browser needed. The DNS part:

"dns": {
  "mxProvider": "Google Workspace",
  "mxHosts": ["aspmx.l.google.com", "alt1.aspmx.l.google.com", "…"],
  "dnsHost": "Cloudflare",
  "spfIncludes": ["_spf.google.com", "spf.mtasv.net", "44092442.spf10.hubspotemail.net", "amazonses.com"],
  "emailAuth": { "spf": true, "spfPolicy": "softfail", "dmarc": true, "dmarcPolicy": "reject" },
  "services": [
    { "name": "Google Workspace", "category": "Email hosting",        "evidence": "MX aspmx.l.google.com" },
    { "name": "Postmark",         "category": "Email delivery",       "evidence": "SPF include:spf.mtasv.net" },
    { "name": "HubSpot",          "category": "Marketing automation", "evidence": "SPF include:44092442.spf10.hubspotemail.net" },
    { "name": "Amazon SES",       "category": "Email delivery",       "evidence": "SPF include:amazonses.com" },
    { "name": "Microsoft 365",    "category": "Email hosting",        "evidence": "TXT MS=…" },
    { "name": "Slack",            "category": "Collaboration",        "evidence": "TXT slack-domain-verification=…" },
    { "name": "Notion",           "category": "Collaboration",        "evidence": "TXT notion-domain-verification=…" },
    { "name": "Stripe",           "category": "Payments",             "evidence": "TXT stripe-verification=…" }
  ]
}

(8 of 19 services shown. The others included Zoom, DocuSign, LaunchDarkly, MongoDB Atlas, Rippling and a few AI vendors.) Verification values are cut at the = on purpose; the API never returns the tokens themselves.

The rest of the same response:

"name": "Linear",
"description": "Purpose-built for planning and building products with AI agents.",
"foundedYear": 2019,
"primaryEmail": "hello@linear.app",
"socialProfiles": { "x": "https://x.com/linear", "github": "https://github.com/linear", "youtube": "https://www.youtube.com/@linear" },
"domainRegistration": { "registrar": "CloudFlare, Inc.", "createdAt": "2018-05-09T23:45:22.384Z", "domainAgeYears": 8.3, "source": "rdap" },
"companySize": { "careersPage": "https://linear.app/careers", "ats": "ashby", "openJobs": 30, "hiring": true },
"evidence": { "foundedYear": { "source": "page text", "url": "https://linear.app/about" } }

So from one domain: email on Google Workspace, transactional mail through Postmark and SES, marketing mail through HubSpot, DMARC at reject, Next.js on Cloudflare, 30 open jobs on Ashby. Each field says where it came from.

How to read it without fooling yourself

This is where most "tech stack" data goes wrong, so I want to be explicit:

MX beats TXT for "who hosts their email". linear.app has a Microsoft MS= verification record and Google MX records. Mail goes to Google. The Microsoft record only proves someone verified the domain with Microsoft at some point. Use mxProvider for the email host; treat TXT-based services as "has (or had) an account".
Self-hosted MX is not the whole story. basecamp.com runs its own MX (mx1.basecamp.com), so mxProvider is "self-hosted or other". But the API also found a Google DKIM key (google._domainkey) and Mailchimp in SPF — they still send some mail through Google and Mailchimp.
Absence is not proof. A tool that never needed DNS verification, or a site that hides its framework, will not show up.
Headers can be rewritten. The detector only trusts Cloudflare headers when the site's DNS actually points at Cloudflare, but anything read from a page is a hint with evidence, not a fact.

Turn a domain list into a CSV

import csv, requests

HOST = "company-enrichment-api13.p.rapidapi.com"
HEADERS = {"X-RapidAPI-Key": "YOUR_RAPIDAPI_KEY", "X-RapidAPI-Host": HOST}
domains = ["linear.app", "basecamp.com", "posthog.com"]

with open("companies.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.writer(f)
    w.writerow(["domain", "name", "email_host", "senders", "frameworks", "hosting", "ats", "open_jobs"])
    for d in domains:
        c = requests.get(f"https://{HOST}/api/v1/company/enrich", params={"domain": d},
                         headers=HEADERS, timeout=60).json()
        if not c.get("success"):
            w.writerow([d, c["error"]["code"]]); continue        # parked_domain, target_unreachable...
        dns = c.get("dns") or {}
        senders = [s["name"] for s in dns.get("services", [])
                   if s["category"] in ("Email delivery", "Email marketing", "Marketing automation")]
        cats = (c.get("techStack") or {}).get("categories", {})
        size = c.get("companySize") or {}
        w.writerow([d, c.get("name"), dns.get("mxProvider"), "; ".join(senders),
                    "; ".join(cats.get("JavaScript framework", [])), "; ".join(cats.get("Hosting / PaaS", [])),
                    size.get("ats"), size.get("openJobs")])

Parked or for-sale domains come back as 422 parked_domain instead of a made-up company. There is also /api/v1/company/enrich/ai, which adds an AI industry label, business model (for example nonprofit) and a one-paragraph summary; it is billed separately, so the plain endpoint never runs a model.

Price

RapidAPI plans as of 1 October 2026:

	Free	Pro	Ultra	Mega
Tech Stack Detector	100 lookups	$9.99 / 2,500	$29.99 / 12,000	$79.99 / 50,000
Company Enrichment	100 lookups + 25 AI	$14.99 / 5,000 + 1,500 AI	$49.99 / 25,000 + 6,000 AI	$129.99 / 100,000 + 20,000 AI

Free plans have hard limits. Overage on paid plans is listed on each pricing page. RapidAPI charges its own bandwidth fee above 10 GB a month.

Tech Stack Detector: https://rapidapi.com/tidytools/api/tech-stack-detector2
Company Enrichment: https://rapidapi.com/tidytools/api/company-enrichment-api13

For lists of thousands of domains without writing code, both are also Apify Actors: https://apify.com/tidytools/website-tech-stack-detector ($4 per 1,000 sites with at least one detection) and https://apify.com/tidytools/company-website-enrichment ($6 per 1,000 companies, includes AI).

Responsible use

Everything here is public: DNS records anyone can query, and pages the company publishes. Still, use it for B2B research, not to profile individuals, and follow the anti-spam rules where you live before you email anyone you found this way.

Disclosure: I built these APIs and Actors and I earn money when people use them. Outputs are real responses from 1 October 2026; the companies were picked as well-known public examples and have no connection to me.

An on-page SEO audit API: a 0–100 score and a fix list in about 2 seconds

Tidy Tools — Sat, 03 Oct 2026 03:52:28 +0000

I wanted an SEO check I could run inside a CI job: one URL in, a number out, and a list of things to fix. No dashboard, no crawl that takes ten minutes, no AI guessing. Rules only, so the same page gives the same score every time.

So I built one. Here is what it returned for two well-known sites on 1 October 2026 (real calls, real responses), and how to wire it into a deploy pipeline.

One call

curl --request GET \
  --url 'https://seo-audit-api15.p.rapidapi.com/api/v1/seo/audit?url=https://htmx.org/' \
  --header 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  --header 'X-RapidAPI-Host: seo-audit-api15.p.rapidapi.com'

1.32 s later:

{
  "success": true,
  "url": "https://htmx.org/",
  "statusCode": 200,
  "renderedWith": "http",
  "score": 90,
  "grade": "A",
  "summary": { "critical": 0, "warnings": 1, "notices": 5, "total": 6 },
  "categoryScores": {
    "indexing": 100, "technical": 100, "meta": 97, "headings": 85, "content": 100, "links": 100,
    "images": 97, "social": 94, "structured-data": 97, "performance": 100, "mobile": 100, "international": 100
  },
  "issues": [
    {
      "severity": "warning",
      "code": "h1-multiple",
      "category": "headings",
      "message": "The page has 2 H1 headings.",
      "fix": "Keep a single main H1 and turn the others into H2 headings."
    },
    {
      "severity": "notice",
      "code": "description-long",
      "category": "meta",
      "message": "The meta description is 338 characters (~2168px) and will likely be cut (about 920px fit).",
      "fix": "Shorten the meta description to about 160 characters."
    },
    {
      "severity": "notice",
      "code": "img-dimensions-missing",
      "category": "images",
      "message": "44 of 44 images have no width/height attributes.",
      "fix": "Add width and height attributes to images so the layout does not shift while they load (CLS)."
    }
  ]
}

(Three of the six issues shown; the other three were missing Open Graph tags, no twitter:card, and no structured data.)

The two H1s are "Platinum Sponsor" and "Gold Sponsor" — exactly the kind of thing nobody notices until a tool points at it. The response also includes the details behind each finding: the full heading outline, title and description pixel widths, the robots.txt rule that matched Googlebot (Allow: /), and the sitemap it found through robots.txt (188 URLs, and it lists this page).

Checking links too

Add checkLinks=10 and it also requests up to 10 links from the page (internal first) and follows their redirects. For https://svelte.dev/ (1.55 s, score 88 / B):

"links": {
  "total": 121, "internal": 16, "external": 105, "unique": 70,
  "checked": { "requested": 10, "checked": 10, "broken": 0, "redirected": 6 }
}

Six of the ten navigation links redirect — /docs/svelte → /docs/svelte/overview, /playground → /playground/hello-world?show=input, and so on — so the audit adds a notice to link to the final URLs. It also flagged a missing canonical, no XML sitemap (none in robots.txt, /sitemap.xml is 404), a 34-character meta description, and 6 render-blocking stylesheets.

None of these are disasters; both sites are in good shape. That is the point of a score: you can see that a page is fine, and see the moment it gets worse.

What it checks

Indexing: status code, redirects, canonical (and whether its target works), meta robots and X-Robots-Tag, robots.txt verdict for Googlebot, sitemap.
Meta and headings: title and description length in characters and pixels, H1 count, skipped levels, the full outline.
Content and images: word count, readability, alt text (empty alt="" counts as decorative, not missing), missing width/height.
Social and structured data: Open Graph, Twitter cards, JSON-LD and Microdata validated against rich-result rules.
Speed hints from the response itself: HTML size, compression, cache headers, render-blocking resources. (This is not Lighthouse; there are no lab Core Web Vitals here.)
Mobile and international: viewport, zoom, lang, hreflang.

It uses a plain HTTP request first and only opens a real browser if the site blocks the request or the HTML is an empty JavaScript shell.

Errors are explicit. A page that does not exist is not "score 0":

{ "success": false, "error": { "code": "target_error", "message": "The page does not exist (HTTP 404).",
  "details": { "targetStatus": 404, "finalUrl": "https://htmx.org/this-page-does-not-exist" } } }

Use it as a CI gate

Fail the deploy if a key page drops below a score, or if any critical issue appears:

# seo_gate.py — run after deploying a preview
import os, sys, requests

HOST = "seo-audit-api15.p.rapidapi.com"
PAGES = ["https://staging.example.com/", "https://staging.example.com/pricing"]
MIN_SCORE = 85

failed = False
for url in PAGES:
    r = requests.get(f"https://{HOST}/api/v1/seo/audit", params={"url": url, "checkLinks": 20},
                     headers={"X-RapidAPI-Key": os.environ["RAPIDAPI_KEY"], "X-RapidAPI-Host": HOST}, timeout=60)
    a = r.json()
    if not a.get("success"):
        print(f"FAIL {url}: {a['error']['code']}"); failed = True; continue
    critical = [i for i in a["issues"] if i["severity"] == "critical"]
    print(f"{a['score']:>3} {a['grade']} {url}")
    for i in a["issues"]:
        print(f"    [{i['severity']}] {i['message']} -> {i['fix']}")
    if a["score"] < MIN_SCORE or critical:
        failed = True

sys.exit(1 if failed else 0)

# .github/workflows/seo.yml (excerpt)
- name: SEO gate
  run: python seo_gate.py
  env:
    RAPIDAPI_KEY: ${{ secrets.RAPIDAPI_KEY }}

Note that the API has to reach the page, so staging must be publicly reachable; private and local addresses are refused on purpose.

Price

RapidAPI plans as of 1 October 2026:

Plan	Per month	Audits	Overage
Basic	$0	200 (hard limit)	—
Pro	$7.99	10,000	$0.001 each
Ultra	$24.99	50,000	$0.0008 each
Mega	$69.99	200,000	$0.0005 each

The free plan covers a CI check on a handful of pages per deploy for a small project. RapidAPI charges its own bandwidth fee above 10 GB a month.

Try it: https://rapidapi.com/tidytools/api/seo-audit-api15

Need a whole site, not one page? The same rules run as a crawler on Apify, with site-wide checks (duplicate titles, sitemap URLs that fail, internal links to redirects) and an optional PageSpeed measurement: https://apify.com/tidytools/seo-audit-crawler ($5 per 1,000 pages).

Disclosure: I built this API and the Apify crawler and I earn money when people use them. The scores above are what the API returned on 1 October 2026; the sites may have changed since. htmx and Svelte were picked as well-known public examples, not because anything is wrong with them.

Prices updated on 2026-10-08.

Podcast to multilingual subtitles in one step

Tidy Tools — Thu, 01 Oct 2026 10:14:46 +0000

Subtitles in one language are easy to get now: run Whisper, get an SRT. Subtitles in three languages still usually mean three steps: transcribe, split the text into subtitle-sized cues, then translate each cue without breaking the timings. Most translation tools do not know what an SRT file is, so the timestamps get mangled or the cues get merged.

I added a subtitleLanguages option to my audio transcriber on Apify so that the whole thing is one run. Here is a real run on a NASA podcast episode, with the timings and the bill.

The input

Give it the podcast's RSS feed, pick the episode, and list the languages:

{
  "podcastFeeds": ["https://www.nasa.gov/feeds/podcasts/curious-universe"],
  "titleIncludes": ["Launching Soon"],
  "maxEpisodesPerFeed": 1,
  "usePublisherTranscripts": false,
  "subtitleLanguages": ["Spanish", "Traditional Chinese"]
}

podcastFeeds also accepts Apple Podcasts show links. Instead of a feed you can pass direct audio or video URLs (MP3, MP4, M4A, WEBM…) or upload files.
titleIncludes picks episodes by title. Without it you get the newest maxEpisodesPerFeed episodes.
usePublisherTranscripts: false forces a real transcription. By default, if the feed already has the publisher's own transcript (<podcast:transcript>), the Actor uses that and charges no audio minutes.
subtitleLanguages takes up to 5 languages, by name or code (es, ja, zh-TW…).

The run


Episode	NASA's Curious Universe, "Launching Soon: NASA's Roman Space Telescope" (4 min 57 s, 7.5 MB MP3)
Feed	108 episodes; 107 filtered out by the title
Transcript	754 words, English detected automatically, Whisper large-v3-turbo
Subtitles	87 cues in English, 87 in Spanish, 87 in Traditional Chinese
Processing time	75 seconds for transcription and both translations (81 s for the whole run)
Price	5 audio minutes × $0.006 + 5 minutes × 2 languages × $0.002 = $0.05

Every cue in the translated files has exactly the same timestamps as the English one. I compared all 87 start and end times in the three SRT files: identical. The longest cue was 5.6 seconds.

Two cues from the three files:

00:00:19,400 --> 00:00:22,740
the latest in a legacy of exploration.

00:00:19,400 --> 00:00:22,740
el último de una larga
tradición de exploración.

00:00:19,400 --> 00:00:22,740
這是探索遺產中的最新成果。

00:00:23,840 --> 00:00:25,160
First, there was Hubble.

00:00:23,840 --> 00:00:25,160
Primero, estuvo Hubble.

00:00:23,840 --> 00:00:25,160
首先，有哈勃。

One thing to watch: "Traditional Chinese" gives you Traditional characters, but word choices can follow mainland usage. Hubble became 哈勃, while Taiwan usually writes 哈伯. And one proper name came out half-translated: "Discovery" (the Space Shuttle) became 迪斯covery. Machine-translated subtitles are a strong first draft, not a finished product: if your audience is in one specific region, have a native speaker check names and terms.

What you get back

One dataset row per episode:

text, segments (with start and end times) and paragraphs
with "wordTimestamps": true, a words array (word, start, end) in every segment, at no extra cost
srtUrl and vttUrl for the original language
translatedSubtitles: for each language, srtUrl, vttUrl, the number of cues and whether it succeeded
episode metadata from the feed: podcastTitle, episodeTitle, pubDate, episodeUrl

Subtitles are cut to subtitle size using word timings: at most 7 seconds and two lines per cue. Raw Whisper segments can run for 20+ seconds, which video platforms reject.

How accurate and how fast?

On longer test files (in the same week, from our own tests):

Compared with NASA's official captions on two ScienceCasts videos, the median start-time error was 0.40–0.51 seconds.
A 62-minute file took 144 seconds, so roughly 17–26× faster than real time.
Whisper sometimes skips a stretch of speech in the middle of a long file. The Actor now re-transcribes any gap of 5 seconds or more without text; on the 62-minute file that recovered 9 passages.

Cost

	Price
Transcription	$0.006 per audio minute ($0.36 per hour)
Speaker labels (optional)	$0.015 per minute instead of $0.006
Translated subtitles	$0.002 per audio minute per language, only for languages that succeed
SRT/VTT, paragraphs, segment and word timestamps	included

So a 30-minute episode in Spanish and German is $0.18 for the transcript plus $0.12 for the two subtitle languages. No start fee.

Schedule it

For a podcast that releases a new episode every week, turn on newEpisodesOnly and give the schedule a monitorName. The Actor remembers which episodes it already transcribed and only takes episodes released after the newest one it has done, so a scheduled run on a day without a new episode transcribes, and charges, nothing. Old episodes are left alone unless you also set backfillOlderEpisodes: true, which works through the back catalogue maxEpisodesPerFeed episodes per run.

{
  "podcastFeeds": ["https://www.nasa.gov/feeds/podcasts/curious-universe"],
  "maxEpisodesPerFeed": 1,
  "newEpisodesOnly": true,
  "monitorName": "curious-universe",
  "subtitleLanguages": ["Spanish", "Japanese"]
}

Or call it from code (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("tidytools/audio-transcriber").call(run_input={
    "podcastFeeds": ["https://www.nasa.gov/feeds/podcasts/curious-universe"],
    "maxEpisodesPerFeed": 1,
    "subtitleLanguages": ["Spanish", "Japanese"],
})
for ep in client.dataset(run["defaultDatasetId"]).iterate_items():
    if not ep.get("success"):
        print("failed:", ep.get("input"), ep.get("error"))
        continue
    print(ep["episodeTitle"], ep["srtUrl"])
    for sub in ep.get("translatedSubtitles", []):
        print(" ", sub["language"], sub.get("srtUrl"))

Or with plain HTTP, which waits for the run and returns the rows (fine for short episodes; use the asynchronous run endpoint for long ones):

curl -X POST "https://api.apify.com/v2/acts/tidytools~audio-transcriber/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"podcastFeeds": ["https://www.nasa.gov/feeds/podcasts/curious-universe"], "maxEpisodesPerFeed": 1, "subtitleLanguages": ["Spanish"]}'

It does not download from YouTube, Vimeo or Loom pages: those links come back as a free error row explaining how to get the file. Use files and feeds you have the right to process.

Try it here: Audio & Video to Text Transcription - Whisper, Podcasts, SRT

Disclosure: I built this Actor and earn money when people run it on Apify.

DEV Community: Tidy Tools

Turn a website list into a B2B contact sheet with Apify and its API

Prepare a list worth running

Run it in Apify Console

Actual example output

Run the same job through the API

Turn rows into a reviewable sheet

Pricing explanation

Recover deleted pages and old URLs with the Wayback Machine, in bulk

Job 1: every URL a site ever had

Job 2: an archive link for every dead link

Job 3: every version of a page, with the text

Job 4: is this expired domain what it claims?

What it costs

Limits worth knowing

Terms and responsible use

Bulk WHOIS with RDAP: domain age, expiry and SPF/DMARC grade for 1,000 domains

WHOIS is being replaced by RDAP. That is good news.

One row per domain

The e-mail security grade

Use case 1: an expiry report for a domain portfolio

Use case 2: an e-mail deliverability audit for clients

Use case 3: a first pass on domain name ideas

What registries actually publish

What it costs

Content Checker alternative on Apify: 100 pages a day for $6/month

Migrate in one minute

Then watch many pages in one run

What a change looks like

Content Checker vs. this Actor

What it costs

Set it up in three steps

Export a trade show exhibitor list to CSV (FABTECH example), then enrich it

Step 1: find the directory URL

Step 2: export the list

What two real runs cost

Step 3: add company data to every row

What it costs

Limits worth knowing

Responsible use

Clean article text + a TL;DR from any news URL in one API call

Step 1: just the article (no AI)

Step 2: the same page, with a TL;DR

Bullets, in another language

Step 3: what happens with a page that is not an article

Python: summarise a list of links

Limits

Price

Track Shopify competitor prices daily with products.json (no API key)

What the feed gives you (and what it does not)

One clean row per product

The real test run

Turn it into a daily price monitor

Bonus: which of these sites run on Shopify?

What it costs

Limits worth knowing

Terms and responsible use

Clean an email list without SMTP pings: typos, disposable and dead domains

What you can know without touching the mailbox

A real run

Validate a scraped lead list in one pipeline

What it costs

What it cannot tell you

Responsible use

I benchmarked my website-to-Markdown crawler against Apify's Website Content Crawler on 5 real sites

The setup

Price and speed

Content quality (first run, before any changes)

What I changed after seeing this

Where WCC is still the better choice

Known issues on my side

Reproduce it

What does this company use? Tech stack + email provider from a domain

Part 1: the website — techstack

Part 2: the domain — DNS tells you the tools behind the website

How to read it without fooling yourself

Turn a domain list into a CSV

Price

Responsible use

An on-page SEO audit API: a 0–100 score and a fix list in about 2 seconds

Part 1: the website — `techstack`