DEV Community: Hay Equipos

How to take full page screenshots of many websites at once

Hay Equipos — Fri, 02 Oct 2026 22:14:51 +0000

You need screenshots of 200 landing pages for a client report, or a dated visual record of competitor pages every month, or a quick check of how your site renders on mobile after a release. Taking them one at a time in a browser is tedious, the cookie banner covers half of every image, and lazy loaded images show up as blank boxes.

The Website Screenshot API: Bulk Full Page PNG and JPEG, published by Hay Equipos on the Apify Store, takes a list of URLs and returns one screenshot per page, with a direct download link, the page title, the HTTP status and the image size. It runs a real headless Chrome browser, scrolls through each page so lazy images load, and hides common cookie banners before it captures.

What you get back

Each image is saved in the run's key value store, and the dataset gets one row per URL. Illustrative example:

{
  "url": "https://www.example.com/",
  "finalUrl": "https://www.example.com/",
  "statusCode": 200,
  "title": "Example Company: Home",
  "screenshotUrl": "https://api.apify.com/v2/key-value-stores/<storeId>/records/screenshot_0001_www_example_com_.jpg?signature=<signature>",
  "screenshotKey": "screenshot_0001_www_example_com_.jpg",
  "format": "jpeg",
  "device": "desktop",
  "viewportWidth": 1280,
  "viewportHeight": 800,
  "fullPage": true,
  "imageWidth": 1280,
  "imageHeight": 3420,
  "truncated": false,
  "bytes": 284100,
  "loadTimeMs": 6150,
  "takenAt": "2026-09-27T06:17:50.332Z",
  "error": null
}

screenshotUrl is a signed link that opens the image directly, without an API token, so you can paste it into a sheet or a report.

url	statusCode	device	imageWidth	imageHeight	truncated	error
https://www.example.com/	200	desktop	1280	3420	false	null
https://shop.example.com/	200	desktop	1280	10000	true	null
https://down.example.org/	null	desktop	null	null	false	net::ERR_NAME_NOT_RESOLVED

Pages that fail still get a row with error filled in, so you know which ones to retry.

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free.
Add your pages to Page URLs, one per line. A missing https:// is added for you, and duplicates are dropped.
Pick a Device: desktop (1280 by 800), laptop (1440 by 900), tablet (820 by 1180 at 2x) or mobile (390 by 844 at 3x, with an iPhone user agent). You can override the size with a custom viewport width and height.
Leave Full page on to capture the whole scrollable page, or turn it off for the first screen only. Maximum page height (default 10,000 CSS pixels) cuts very long pages.
Choose Image format: PNG (lossless) or JPEG (much smaller, with a quality setting from 1 to 100).
Adjust Wait until (DOM ready, load event, or network idle for heavy single page apps) and Extra wait before capture (default 1,000 ms).
Keep Hide cookie banners and Scroll to load lazy images on. Add your own CSS selectors to Also hide these elements for chat widgets or newsletter popups. Turn on Dark mode if you want the page's dark theme.
Click Start. Open the Output tab and click any screenshotUrl to view or download the image.

Calling it from code

The actor id is pistachio_implementation/website-screenshot-api. Keep your token in APIFY_TOKEN.

curl:

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~website-screenshot-api/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://apify.com"], "device": "mobile", "format": "jpeg"}'

Python with apify-client, downloading every image to a local folder:

import os
import urllib.request
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("pistachio_implementation/website-screenshot-api").call(
    run_input={
        "urls": ["https://apify.com", "https://en.wikipedia.org/wiki/Web_scraping"],
        "device": "desktop",
        "fullPage": True,
        "format": "jpeg",
        "jpegQuality": 80,
        "hideCookieBanners": True,
    }
)

os.makedirs("shots", exist_ok=True)
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["error"]:
        print("failed:", row["url"], row["error"])
        continue
    urllib.request.urlretrieve(row["screenshotUrl"], os.path.join("shots", row["screenshotKey"]))

Pricing

Pay per event, with no subscription and no platform usage charges on top:

$3.00 per 1,000 screenshots ($0.003 each).
Failed pages (DNS errors, timeouts after one retry) are free.

A page that loads and is captured counts as a screenshot, whatever its HTTP status, so check statusCode and title if you capture pages that may be missing or blocked.

Limits and what it does not do

No login. Pages behind a login cannot be captured, and the actor never logs in.
Sites with strong bot protection may show their challenge page instead of the content. You then get a screenshot of the challenge, and the status code and title help you spot it.
Cookie banner hiding covers the most common consent tools (such as OneTrust, Cookiebot, Usercentrics, Didomi and Quantcast) by hiding them with CSS. It never clicks "accept", and it does not catch every custom popup.
Pages longer than Maximum page height are cut and marked truncated. Images are also kept under about 16,000 device pixels tall, so on the mobile preset (3x) a full page capture stops at about 5,300 CSS pixels.
Speed: two pages at once by default. Light pages take a few seconds, heavy full page captures 20 to 40 seconds at the default 1 GB of memory. You can raise Pages at once up to 5, but above 2 you should give the run 2 GB or more.
PNG and JPEG only. There is no PDF output in this version.
Images live in the run's storage for your Apify plan's retention period. Download them if you need to keep them longer.

Try it on the Apify Store: https://apify.com/pistachio_implementation/website-screenshot-api

How to download Trustpilot reviews and TrustScore to a spreadsheet

Hay Equipos — Fri, 02 Oct 2026 22:14:15 +0000

You want to know what customers say about your company, or about five competitors, and Trustpilot holds the answer in hundreds of reviews spread across dozens of pages. Copying them into a sheet by hand takes an afternoon, and you would have to do it again next month. What you need is a table: one row per review, plus one row per company with its TrustScore and star breakdown.

The Trustpilot Reviews and TrustScore Scraper, published by Hay Equipos on the Apify Store, takes a list of company domains and returns exactly that. Reviewer names, profile pictures and profile links are never collected, so you get the opinion without the personal data.

What you get back

Every row has a type field: review, company or error. The dataset has two ready made views (Reviews and Company summaries) that you can download separately.

A review row (illustrative values):

{
  "type": "review",
  "companyDomain": "www.example-shop.com",
  "companyName": "Example Shop",
  "reviewId": "66f0c2a1b3e4d5f6a7b8c9d0",
  "reviewUrl": "https://www.trustpilot.com/reviews/66f0c2a1b3e4d5f6a7b8c9d0",
  "rating": 2,
  "title": "Slow delivery",
  "text": "The order arrived nine days late and support took a week to answer.",
  "language": "en",
  "reviewerCountry": "US",
  "publishedDate": "2026-09-20T10:12:00.000Z",
  "experiencedDate": "2026-09-08T00:00:00.000Z",
  "isVerified": true,
  "likes": 0,
  "companyReply": {
    "text": "Sorry about the delay, we have contacted you by email.",
    "publishedDate": "2026-09-22T14:30:00.000Z"
  }
}

A company summary row:

companyDomain	trustScore	totalReviews	oneStar	fiveStars	replyPercentage
`www.example-shop.com`	3.9	1240	180	760	42
`www.example-rival.com`	2.1	515	290	120	0

The summary row also includes reviewsLast12Months, the full star distribution, reviewsMatchingFilters, review counts per language, categories, country code, whether the profile is claimed, and the average days to reply. A domain with no Trustpilot page comes back as an error row instead of breaking the run.

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free.
In Company domains, add one domain per line, written as Trustpilot lists it. For trustpilot.com/review/www.bestbuy.com, that is www.bestbuy.com. Pasting the whole Trustpilot link also works.
Set Maximum reviews per company (default 100, newest first). Set it to 0 if you only want the TrustScore summary.
Keep Include company summary on to get one summary row per company.
Optional filters: Review language (a two letter code such as en, or all), Star ratings, Date range (last 30 days, 3, 6 or 12 months), Verified reviews only and Reviews with a company reply only.
Click Start, then export from the Output tab as CSV, Excel or JSON.

Example input that pulls recent complaints for two companies:

{
  "companyDomains": ["www.bestbuy.com", "www.zappos.com"],
  "maxReviewsPerCompany": 200,
  "stars": ["1", "2"],
  "dateRange": "last6months"
}

Calling it from code

The actor id is pistachio_implementation/trustpilot-reviews-company-scores. Store your token in APIFY_TOKEN.

curl, run and get results in one request:

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~trustpilot-reviews-company-scores/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"companyDomains": ["www.zappos.com"], "maxReviewsPerCompany": 50}'

Python with apify-client:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("pistachio_implementation/trustpilot-reviews-company-scores").call(
    run_input={
        "companyDomains": ["www.bestbuy.com", "www.zappos.com"],
        "maxReviewsPerCompany": 0,  # summaries only
        "includeCompanySummary": True,
    }
)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["type"] == "company":
        print(row["companyDomain"], row["trustScore"], row["totalReviews"])
    elif row["type"] == "error":
        print("skipped", row["companyDomain"], row["error"])

Schedule that script weekly and you have a TrustScore tracker for your brand and your competitors.

Pricing

Pay per event, no subscription:

$3.00 per 1,000 reviews ($0.003 per review)
$0.005 per company summary
Error rows (company not on Trustpilot, page could not load) are free

You can set a maximum cost per run in the run options. The actor stops cleanly when that amount is reached.

Limits and what it does not do

Trustpilot shows visitors who are not logged in at most 10 pages (200 reviews) of any one listing. The actor never logs in. To reach further it reads several listings for the same company (each star level, each review language, verified only, replied only) and removes duplicates. For small and mid sized companies you normally get every review. For very large companies you get the newest reviews across star levels and languages, not the full history. In one test it collected 1,094 unique reviews for a company with over 7,000.
Speed is roughly 200 to 300 reviews a minute, with polite pauses between pages. It runs a real headless browser, so it needs at least 1 GB of memory.
Review text is kept as written and may itself mention names. Only the reviewer identity fields are left out.
When a reviewer leaves the title empty, Trustpilot builds one from the first words of the text, so some titles end in "…". The full text is always in text.
Only public pages are read. No account, no login. A proxy option exists but is not needed in most cases.

Please use the reviews you collect in line with Trustpilot's terms.

Try it on the Apify Store: https://apify.com/pistachio_implementation/trustpilot-reviews-company-scores

How to export posts from a public Telegram channel to CSV or JSON

Hay Equipos — Fri, 02 Oct 2026 22:05:27 +0000

You want the posts from a public Telegram channel in a spreadsheet: what was posted, when, and how many people saw it. Telegram's official API needs a phone number, an app registration and a client library, which is a lot of setup for "give me the last 500 posts as CSV". Copying posts by hand stops working after the first dozen.

The Telegram Channel Scraper actor, published by Hay Equipos on the Apify Store, reads the public web preview that Telegram itself serves at t.me/s/<channel> (the same page anyone can open in a browser) and walks back through the channel history page by page until it reaches your date or post limit. No Telegram account, no phone number, no API key.

What you get back

One row per post. A trimmed example (values are illustrative):

{
  "channel": "examplechannel",
  "channelTitle": "Example Channel",
  "channelSubscribers": 125000,
  "channelVerified": false,
  "postId": 4821,
  "postUrl": "https://t.me/examplechannel/4821",
  "date": "2026-09-14T08:15:00.000Z",
  "edited": false,
  "text": "Weekly market recap is out. #markets #recap",
  "views": 48300,
  "reactions": [{ "emoji": "👍", "count": 310 }, { "emoji": "🔥", "count": 95 }],
  "reactionsTotal": 405,
  "mediaTypes": ["photo"],
  "photoUrls": ["https://cdn4.telesco.pe/file/..."],
  "videoCount": 0,
  "documentNames": [],
  "poll": null,
  "links": ["https://example.com/recap"],
  "hashtags": ["#markets", "#recap"],
  "linkPreview": null,
  "forwardedFrom": null,
  "isServiceMessage": false,
  "scrapedAt": "2026-09-27T06:09:41.563Z"
}

Each row also carries videoThumbnailUrl and forwardedFromUrl. Polls come with their question and option percentages, and link previews come with site name, title and description. A RUN_SUMMARY record in the run's key value store lists every channel with its status, title, subscriber count, description, and how many posts were read and saved.

In table form, the columns most people keep:

postUrl	date	views	reactionsTotal	text
https://t.me/examplechannel/4821	`2026-09-14`	48300	405	Weekly market recap is out...
https://t.me/examplechannel/4820	`2026-09-13`	51200	388	New listing announced...

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free. You need a free Apify account.
In Public channels, add one channel per line. A username (telegram), @username, or a link (https://t.me/durov or https://t.me/s/durov) all work.
Set Maximum posts per channel (default 100, newest posts first) and, if you like, Maximum posts in total (default 10,000).
To limit the date range, fill Posted on or after and Posted before, for example 2026-09-01. Reading stops once the actor reaches posts older than your start date.
To keep only posts about a topic, add words to Text contains any of. Matching ignores case.
Leave Include service messages off unless you want notices like a changed channel photo or a pinned message.
Click Start. When the run finishes, open the Output tab and export as CSV, Excel, JSON or HTML.

Example input:

{
  "channels": ["telegram", "https://t.me/durov"],
  "maxPostsPerChannel": 200,
  "postedAfter": "2026-06-01",
  "keywords": []
}

Calling it from code

The actor id is pistachio_implementation/telegram-channel-posts. Keep your token in the APIFY_TOKEN environment variable.

With curl, this runs the actor and returns the dataset items in one call:

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~telegram-channel-posts/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"channels": ["telegram"], "maxPostsPerChannel": 50}'

With Python and the apify-client package (pip install apify-client):

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("pistachio_implementation/telegram-channel-posts").call(
    run_input={
        "channels": ["telegram", "durov"],
        "maxPostsPerChannel": 100,
        "postedAfter": "2026-09-01",
        "keywords": ["update"],
    }
)

for post in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(post["postUrl"], post["date"], post["views"])

To monitor a channel, schedule the actor daily with postedAfter set to yesterday, or store the highest postId from the last run and compare.

Pricing

Pay per event, with no subscription and no platform usage charges on top:

$0.30 per 1,000 posts saved ($0.0003 per post).
Posts removed by your date or keyword filters, and channels that cannot be read, cost nothing.

Limits and what it does not do

Public channels only. Private channels, groups, chats and user accounts have no web preview. They are reported as "no public preview" in RUN_SUMMARY instead of failing the whole run. Some public channels also turn their preview off.
Views and reactions are the rounded numbers Telegram shows. "1.48M" becomes 1480000, so treat large counts as approximate.
Custom emoji reactions come back as custom:<id>, because Telegram's page shows them as images.
Media links point to Telegram's CDN and expire after a while. Download the files soon if you need them.
The actor waits 1.5 seconds between pages to stay polite, which works out to roughly 800 posts a minute.
It does not collect post signatures (the admin who posted), member lists, phone numbers or user profiles. It reads only what a public channel shows to any visitor.
It depends on the markup of Telegram's preview page. If Telegram changes that page, the parser needs an update.

Please use the posts you collect in line with Telegram's terms and the rights of the people who wrote them.

Try it on the Apify Store: https://apify.com/pistachio_implementation/telegram-channel-posts

How to find out what technology a website uses, in bulk

Hay Equipos — Fri, 02 Oct 2026 22:04:51 +0000

Sales teams and agencies qualify prospects by technology: every Shopify store using a certain email tool, WordPress sites without a chat widget, companies on a particular CDN. Browser extensions answer the question for one site at a time. When you have a list of 5,000 domains, you need the answer as a table.

Tech Stack Detector by Hay Equipos takes a list of domains and returns, for each one, the CMS, ecommerce platform, analytics and tag managers, payment providers, live chat widgets, JavaScript and web frameworks, CDN, hosting and web server, with versions where the site exposes them.

How it works

For each domain, the actor fetches the public home page and its response headers, plus robots.txt, and matches them against about 3,600 technology fingerprints from an open source, MIT licensed fingerprint database. It checks headers, cookie names, meta tags, script sources, inline scripts, CSS, HTML and page structure, then runs a second pass for technologies that only make sense when another one is present.

No browser is started, which keeps it fast and cheap. The actor uses an honest user agent, respects robots.txt by default, sends no cookies, needs no login, and returns technology names only, with no personal data.

What you get back

One row per domain. Trimmed example with illustrative values:

{
  "domain": "example-shop.com",
  "finalUrl": "https://www.example-shop.com/",
  "statusCode": 200,
  "title": "Example Shop: Outdoor Gear",
  "technologyCount": 11,
  "cms": [],
  "ecommerce": ["Shopify"],
  "analytics": ["Google Analytics"],
  "tagManagers": ["Google Tag Manager"],
  "payments": ["PayPal", "Apple Pay"],
  "liveChat": [],
  "cdn": ["Cloudflare"],
  "security": ["HSTS"],
  "technologies": [
    { "name": "Shopify", "version": null, "confidence": 100, "categories": ["Ecommerce"], "website": "https://www.shopify.com" },
    { "name": "jQuery", "version": "3.6.0", "confidence": 100, "categories": ["JavaScript libraries"], "website": "https://jquery.com" }
  ],
  "possiblyBlocked": false,
  "error": null,
  "checkedAt": "2026-09-27T05:45:12.000Z"
}

Every row has the same summary columns, which makes filtering in a spreadsheet easy: cms, ecommerce, analytics, tagManagers, payments, liveChat, marketingAutomation, crm, advertising, jsFrameworks, webFrameworks, uiFrameworks, jsLibraries, cdn, hosting, webServers, programmingLanguages, security, cookieCompliance, abTesting and reviews. The technologies array holds the full detection list with version, confidence and categories.

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free.
In Domains, paste domains or URLs, one per line. Only the home page of each domain is checked, and duplicates are removed.
Keep Respect robots.txt on. A domain whose robots.txt closes its home page to crawlers is skipped and not charged.
Set Minimum confidence (default 50, from 0 to 100) to control how strict detections are.
Leave Include the full technology list on, or switch it off for a compact table with only the summary columns.
Set Domains checked in parallel (default 5, up to 20) and Request timeout (default 20 seconds). Parallelism is also capped by the run's memory, so give large runs more memory if you want more checks at once.
Click Start, then export to CSV or Excel and filter on the summary columns.

Calling it from code

With curl:

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~tech-stack-detector/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "domains": ["example-shop.com", "example.org"],
    "minConfidence": 50,
    "includeAllTechnologies": false
  }'

With Python and the apify-client package, finding Shopify stores in a lead list:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

domains = ["example-shop.com", "example.org", "store.example.net"]

run = client.actor("pistachio_implementation/tech-stack-detector").call(
    run_input={
        "domains": domains,
        "maxConcurrency": 10,
        "includeAllTechnologies": True,
    }
)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["error"]:
        print("skipped", row["domain"], row["error"])
    elif "Shopify" in row["ecommerce"]:
        print(row["domain"], "payments:", row["payments"], "chat:", row["liveChat"])

Pricing

Pay per event, with no subscription and no platform usage charge on top:

Event	Price
Actor start (Apify's standard start event)	$0.00005 per run
Domain analyzed	$0.002 per domain ($2 per 1,000 domains)

A domain is charged only when the site actually served its home page with a status below 400. Domains that do not resolve, time out, answer with an error or bot wall status, or are skipped because of robots.txt are free. You can set a maximum charge per run and the actor stops when it is reached.

Limits and what it does not do

Static analysis only. Tools injected later by a tag manager or loaded on scroll (analytics loaded through a tag manager, many chat widgets) do not appear in the page source and may be missed. The tag manager itself is detected. This is the most common reason a technology you know about is missing.
Home page only. A shop on a subdomain such as shop.example.com should be entered separately.
Bot walls. Some large sites block every automated visitor. Those rows come back with the HTTP status, possiblyBlocked: true and whatever could be read from the headers.
Fixed fingerprint snapshot. The fingerprints come from a release dated January 2023. Technologies launched since then may be missing or named differently.
No DNS or TLS certificate checks.

The actor reads only public pages. Use the results in line with each site's terms.

Try it on the Apify Store: https://apify.com/pistachio_implementation/tech-stack-detector

How to track a competitor's Shopify prices and new products

Hay Equipos — Fri, 02 Oct 2026 21:59:04 +0000

If you run a DTC brand, a dropshipping store or a reseller business, you want to know when a competitor or supplier changes a price, puts something on sale, runs out of a size or launches a product. Checking their store by hand every morning does not scale, and a plain scraper only gives you today's catalog, not what changed since yesterday.

Shopify Catalog and Price Change Monitor by Hay Equipos reads a Shopify store's public product feed, saves a snapshot, and on every later run tells you exactly what changed.

How it works

It reads the store's public /products.json feed, the same feed Shopify serves to every visitor, plus /meta.json for the store name and currency. No login, no cookies, no cart or checkout pages.
It saves the catalog after each run in a named key value store in your own Apify account (default name shopify-catalog-monitor-snapshots, one record per store).
The next run compares against that snapshot and flags each product as baseline, unchanged, new, updated or removed, with a list of exactly what changed.
One feed page at a time per store, a pause between pages, robots.txt respected by default and retries with backoff when a store rate limits.

What you get back

One row per product, with all its variants. A product whose price changed since the last run (illustrative values, trimmed):

{
  "recordType": "product",
  "store": "www.example-store.com",
  "storeName": "Example Store",
  "currency": "USD",
  "productId": 7340901859408,
  "handle": "trail-runner",
  "title": "Trail Runner",
  "vendor": "Example Store",
  "productType": "Shoes",
  "productUrl": "https://www.example-store.com/products/trail-runner",
  "minPrice": 95,
  "maxPrice": 110,
  "available": true,
  "variantCount": 8,
  "variantsInStock": 5,
  "onSale": true,
  "changeType": "updated",
  "changes": [
    { "field": "price", "variantId": 42146889039952, "variantTitle": "9", "from": 110, "to": 95 },
    { "field": "available", "variantId": 42146889039953, "variantTitle": "10", "from": true, "to": false }
  ],
  "variants": [
    { "variantId": 42146889039952, "title": "9", "sku": "TR-09", "price": 95, "compareAtPrice": 110, "onSale": true, "available": true }
  ],
  "checkedAt": "2026-09-27T05:52:40.000Z"
}

Rows also include tags, imageUrl, imagesCount, createdAt, updatedAt, publishedAt and, if you ask for it, a plain text description. Changes can be a new title, a new price, a new compare at (sale) price, a stock flag flip, or a variant added or removed.

A per store summary (products found, new, updated, removed, whether the whole catalog was read) is saved in the run's key value store under SUMMARY.

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free.
In Shopify store URLs, add one store per line, as a link or a bare domain. Only the domain is used.
Set Maximum products per store (default 1,000). Use 0 for the whole catalog, up to 25,000.
Pick Output: all products with change flags, or only changes since the previous run.
Keep the default Snapshot store name, or use a different name per client to keep separate histories.
Leave Respect robots.txt on. Adjust Stores processed in parallel (default 2) and Pause between pages (default 1,000 ms) if needed.
Click Start. The first run for each store saves the baseline. Changes appear from the second run on.

For daily alerts, save the input as a task with Output set to only changes, add a daily schedule, and connect the dataset to Slack, email, a sheet or a webhook through Apify integrations.

Calling it from code

With curl:

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~shopify-catalog-monitor/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "storeUrls": ["www.example-store.com"],
    "maxProductsPerStore": 0,
    "outputMode": "changes"
  }'

With Python and the apify-client package:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("pistachio_implementation/shopify-catalog-monitor").call(
    run_input={
        "storeUrls": ["www.example-store.com", "shop.example.org"],
        "maxProductsPerStore": 0,
        "outputMode": "changes",
        "snapshotStoreName": "competitor-watch",
    }
)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["recordType"] == "error":
        print("could not read", row["storeUrl"], row["error"])
        continue
    for c in row["changes"]:
        print(row["store"], row["changeType"], row["title"], c)

Pricing

Pay per event, with no subscription and no platform usage charge on top:

Event	Price
Actor start (Apify's standard start event)	$0.00005 per run
Product checked	$0.001 per product read ($1 per 1,000 products)
Change detected	$0.005 per product that is new, updated or removed

Products are charged as checked in both output modes, because every product is read and compared. The first run for a store never charges for changes. Stores that fail, such as password protected stores or stores that block the feed, return an error row and are not charged. You can set a maximum charge per run and the actor stops when it is reached.

Limits and what it does not do

Shopify stores only, and only those that leave the public product feed on.
Removed products are reported only when the whole catalog was read on both runs. If you cap products below the catalog size, new and updated products are still flagged, but removals are held back to avoid false alarms. A capped run can also surface products as new when the feed order shifts, so read whole catalogs when you can.
No inventory quantities. The public feed shows only whether each variant can be bought.
Default currency only. Market specific prices are not included.
Catalogs above 25,000 products are cut at 25,000.

The actor is an independent tool and is not affiliated with Shopify. Use the data in line with each store's terms.

Try it on the Apify Store: https://apify.com/pistachio_implementation/shopify-catalog-monitor

How to export research paper metadata and citation counts to CSV

Hay Equipos — Fri, 02 Oct 2026 21:58:29 +0000

Literature reviews, citation analysis and reference cleanup all start the same way: you need a table of papers with titles, authors, journals, dates and citation counts. Academic search pages are built for reading one result at a time, not for exporting hundreds of records, and copying metadata by hand is slow and error prone.

Research Paper Search by Hay Equipos takes topics or DOIs and returns one structured row per paper. The data comes from Crossref, the registry that issues DOIs for most of the world's publishers, through its open public API. There is no page scraping involved, so runs are steady and there are no captchas.

What you get back

One row per work. Example with illustrative values:

{
  "query": "large language models in education",
  "source": "search",
  "doi": "10.1234/example.2025.0042",
  "doiUrl": "https://doi.org/10.1234/example.2025.0042",
  "title": "Using Large Language Models for Feedback in Higher Education",
  "type": "journal-article",
  "journal": "Journal of Example Education Research",
  "publisher": "Example Academic Press",
  "publishedDate": "2025-03-14",
  "year": 2025,
  "authors": ["A. Author", "B. Author"],
  "authorCount": 2,
  "abstract": "This study examines ...",
  "citationCount": 18,
  "referenceCount": 54,
  "subjects": ["Education"],
  "issn": ["1234-5678"],
  "openLicense": true,
  "funders": [],
  "publisherUrl": "https://journal.example.org/articles/42"
}

Rows also include subtitle, firstAuthorAffiliation, isbn, volume, issue, pages, language, licenseUrls, fullTextLinks and relevanceScore. citationCount is the number of other Crossref works that cite the paper. openLicense is true when the work carries a Creative Commons license. The same paper is never saved twice in one run.

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free. A free Apify account is enough.
Fill Search queries with topics, one per line, or DOIs to look up with DOIs or DOI links. You can use both in one run. Queries match titles, authors, journals and other bibliographic text.
Narrow the results with Published from and Published until (formats 2024, 2024-06 or 2024-06-30) and Work types (journal article, conference paper, book chapter, book, preprint, report, dissertation, dataset and more).
Pick Sort by: relevance, newest first or most cited first.
Optionally turn on Only works with an abstract or Only works with a license, and switch off Include abstracts for smaller rows.
Set Results per query (default 100, up to 10,000) and Maximum papers (default 1,000).
Optionally add Your contact email. Crossref serves requests that include one from a faster pool. It is sent only to Crossref and is not saved in the output.
Click Start, then export the results as CSV, Excel or JSON.

How the sorting works

"Newest first" and "most cited first" do not sort the whole of Crossref. They take the most relevant matches for your query (five times the results you ask for, at least 200 and at most 1,000) and reorder those. This is deliberate: sorting the entire index by citations ignores relevance and returns papers that are off topic.

Calling it from code

With curl, using the synchronous endpoint that returns the rows directly:

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~research-paper-search/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["large language models in education"],
    "fromDate": "2024",
    "types": ["journal-article"],
    "sortBy": "most-cited",
    "maxResultsPerQuery": 50
  }'

With Python and the apify-client package:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("pistachio_implementation/research-paper-search").call(
    run_input={
        "queries": ["CRISPR off target effects"],
        "dois": ["10.1038/nature14539"],
        "fromDate": "2022",
        "onlyWithAbstract": True,
        "sortBy": "relevance",
        "maxResultsPerQuery": 200,
    }
)

for paper in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(paper["year"], paper["citationCount"], paper["title"], paper["doiUrl"])

summary = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("RUN_SUMMARY")
print(summary["value"]["errors"])  # DOIs not found, queries with no matches

Pricing

Pay per event, with no subscription and no platform usage charge on top:

Event	Price
Paper saved	$0.001 per paper ($1.00 per 1,000 papers)

There is no start fee. DOIs that are not found and duplicates are free. For example, 200 papers on a topic cost $0.20, and looking up 1,000 DOIs costs $1.00. You can set a maximum charge per run and the actor stops cleanly when it is reached.

Limits and what it does not do

Abstracts exist only when the publisher deposited one with Crossref. Many older works and some large publishers have none. Use "Only works with an abstract" if you need them.
Citation counts are Crossref's own citation links. They are usually lower than counts from services that also index web pages and theses.
Speed: without a contact email, Crossref's public pool allows about one request per second, and each request returns up to 200 papers. With a contact email it is faster.
Relevance ranking is Crossref's. Results far down a very broad query get looser, so narrow the query or add a date range and type filter.
Up to 10,000 results per query, which is Crossref's paging limit.
No full text download. The actor returns links. Access depends on the publisher and the license.
Only works registered with a DOI are covered. Loose PDFs on the web are not.
No author profiles. Author names appear only as the paper's published byline.

The actor is an independent tool and is not affiliated with Crossref. Use the data in line with Crossref's terms and each publisher's license.

Try it on the Apify Store: https://apify.com/pistachio_implementation/research-paper-search

How to export Redfin for sale and sold home listings to a spreadsheet

Hay Equipos — Fri, 02 Oct 2026 21:52:43 +0000

You are running comps for a property, tracking a market as an investor, or building a proptech prototype, and you need the homes in a ZIP code as rows in a spreadsheet: list price, beds, baths, square feet, year built, and for sold homes the sold price and date. Redfin shows all of that on its map, but copying it by hand is slow, and one search on the site stops at 350 homes.

The Redfin Home Listings Scraper: For Sale and Sold Homes, published by Hay Equipos on the Apify Store, takes ZIP codes, cities or Redfin search URLs and returns one clean row per home. It reads the same public search data the Redfin website loads in your browser. No login, no account. This is an independent tool and is not affiliated with Redfin.

What you get back

One row per home, with the same fields whichever input produced it. Illustrative example:

{
  "source": "redfin",
  "listingUrl": "https://www.redfin.com/TX/Austin/123-Example-St-78704/home/00000000",
  "address": "123 Example St, Austin, TX 78704",
  "city": "Austin",
  "state": "TX",
  "zip": "78704",
  "latitude": 30.2449,
  "longitude": -97.7689,
  "status": "Sold",
  "mlsStatus": "Closed",
  "price": 615000,
  "soldPrice": 615000,
  "soldDate": "2026-09-10",
  "beds": 3,
  "baths": 2,
  "sqft": 1650,
  "lotSizeSqft": 6200,
  "yearBuilt": 1984,
  "daysOnMarket": null,
  "pricePerSqft": 373,
  "hoaPerMonth": null,
  "propertyType": "House",
  "primaryPhotoUrl": "https://ssl.cdn-redfin.com/photo/...",
  "photoCount": 28,
  "isNewConstruction": false,
  "searchLocation": "78704"
}

address	status	price	beds	baths	sqft	soldDate
123 Example St, Austin, TX 78704	Sold	615000	3	2	1650	`2026-09-10`
45 Sample Ave #2, Austin, TX 78704	For sale	489000	2	2	1120

status is normalized to For sale, Coming soon, Pending or Sold, while mlsStatus keeps the wording from the listing source. For homes that are for sale, lastSoldDate shows the previous sale when Redfin has one. Rows also include street, stories, MLS id, property id and listing id.

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free.
In Locations, add one per line: a 5 digit ZIP code (78704), a city with its state code (Austin, TX), or any redfin.com search URL for a city, ZIP, neighborhood or county.
Pick Listing status: For sale (active and coming soon) or Sold.
For sold homes, pick Sold within, from the last week up to the last 5 years (default last 3 months).
Set Max listings per location (default 200). Large areas are split by price automatically to go past the 350 homes per search cap.
Optional filters: property types, min and max price, bedrooms, minimum bathrooms, square feet and year built.
Turn on Include all photo URLs if you want every photo link (up to 100) instead of just the main one.
Click Start, then export from the Output tab as CSV, Excel or JSON.

Tip: if you already set up a search on redfin.com with filters, paste that URL. A Redfin URL that carries its own /filter/ path keeps those filters and ignores the filter fields in the form.

Example input for sold comps:

{
  "locations": ["98103", "Bend, OR"],
  "status": "sold",
  "soldWithinDays": "30",
  "propertyTypes": ["house", "townhouse"],
  "minBeds": 3,
  "maxItemsPerLocation": 100
}

Calling it from code

The actor id is pistachio_implementation/redfin-home-listings-for-sale-sold. Keep your token in APIFY_TOKEN.

curl:

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~redfin-home-listings-for-sale-sold/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"locations": ["78704"], "status": "forSale", "minPrice": 500000, "maxPrice": 1500000}'

Python with apify-client:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("pistachio_implementation/redfin-home-listings-for-sale-sold").call(
    run_input={
        "locations": ["78704", "https://www.redfin.com/city/30818/TX/Austin"],
        "status": "sold",
        "soldWithinDays": "90",
        "propertyTypes": ["house"],
        "maxItemsPerLocation": 300,
    }
)

for home in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(home["address"], home["soldPrice"], home["soldDate"], home["sqft"])

Schedule the run daily or weekly on Apify to track a market over time. The data is read live at the moment the actor runs.

Pricing

Pay per result: $2 per 1,000 listings ($0.002 per home). No start fee, no monthly fee and no platform usage charge on top. You can set a spending limit on a run and the actor stops when it is reached.

Limits and what it does not do

Redfin only. Other large US listing sites were tested and block automated access from cloud servers with bot challenges. Getting past them would take anti bot evasion, which this actor does not do.
Network matters. From some networks, especially outside the US, Redfin answers with a bot check (a captcha page) instead of data. The actor does not try to solve it. It retries with backoff, then reports the location as failed with a clear message. On the Apify platform Redfin answered directly in testing, but a run from a network Redfin challenges will fail rather than return partial guesses.
Days on market is usually empty for sold homes, because Redfin shows it only while a home is listed.
No agent names, brokerage names, phone numbers or emails are collected, and listing descriptions are not copied.
"City, ST" works for cities that Redfin lists on its state pages. For smaller towns, open the town on redfin.com and paste its URL.
Very large searches are split by price at most six times. If a price band is still full after that, the log says so and a few homes may be missing. Add ZIP codes or filters to cover them.
Coverage is Redfin's coverage. Some rural areas have thinner data.
Requests are spaced 2.5 seconds apart, so very large runs take a few minutes.

Please use the listing data in line with Redfin's terms.

Try it on the Apify Store: https://apify.com/pistachio_implementation/redfin-home-listings-for-sale-sold

How to scrape product prices, stock and GTIN from any online store

Hay Equipos — Fri, 02 Oct 2026 21:52:07 +0000

You have a list of product pages from competitors, suppliers or resellers, and you need the price, the stock status and the barcode for each one in a spreadsheet. Copying them by hand works for ten products and falls apart at five hundred. Writing a scraper per store is worse, because every store lays out its pages differently.

There is a shortcut. Most online stores already publish their product data in a machine readable form (Schema.org product markup and Open Graph product tags), because search engines and social previews need it. The Ecommerce Product Page Scraper by Hay Equipos reads that data from any product URL you give it, so the same tool works on Shopify, WooCommerce, BigCommerce, Magento, Wix, Squarespace, PrestaShop and custom stores.

What you get back

One row per URL. Here is an example with illustrative values:

Field	Example
`name`	Classic Canvas Sneaker
`brand`	Example Brand
`price`	64
`currency`	USD
`listPrice`	80
`onSale`	true
`availability`	InStock
`inStock`	true
`sku`	`CCS-WHT-09`
`gtin`	`0123456789012`
`rating`	4.5
`reviewCount`	312
`variantCount`	12
`status`	ok

Rows also carry mpn, lowPrice and highPrice, condition, category, color, seller, images, canonicalUrl, language, dataSources (which markup the data came from), an optional plain text description (up to 5,000 characters) and a variants list with size, color, SKU, GTIN, price and stock per variant.

Every row has a status:

ok: product data found (the only status that is charged)
no_product_data: the page loaded but had no price and no Schema.org product
blocked: the site answered with a bot check or access denied page
http_error: 404, 410, 500 and similar
skipped_robots_txt: the site's robots.txt closes the page to automated tools
error: timeout or network failure

A RUN_SUMMARY record in the run's key value store counts each status.

How it works

The actor fetches each page with a plain HTTP request. It reads JSON-LD product blocks first (including @graph, ProductGroup with variants, AggregateOffer and list price specifications), then fills gaps from microdata, then from Open Graph product: tags. No browser and no AI guessing, so the results are repeatable: the same page gives the same row.

It respects robots.txt by default, never logs in, never solves captchas and does not use proxies to get around blocks. Each site gets one request at a time, with a pause between requests.

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free. You need a free Apify account.
In Product page URLs, paste your links, one per line. Use single product pages, not category or search pages.
Leave Include description and Include variants on, or switch them off for smaller rows.
Keep Respect robots.txt on. Adjust Parallel requests (default 5, across different sites) and Pause between requests to the same site (default 1,000 ms) if you need to.
Set Maximum URLs (default 1,000) as a safety cap, and optionally a maximum charge for the run in the run options.
Click Start. When the run finishes, open the output table and export it as CSV, Excel or JSON.

For price tracking, save the input as a task and add a daily schedule. Then compare price, listPrice and inStock between runs in your sheet or database.

Calling it from code

With curl, using the synchronous endpoint that returns the dataset items directly (good for small lists):

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~product-page-extractor/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "productUrls": [
      "https://www.example-store.com/products/classic-canvas-sneaker",
      "https://shop.example.com/p/desk-lamp-black"
    ],
    "includeDescription": false,
    "includeVariants": true
  }'

With Python and the apify-client package (pip install apify-client):

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("pistachio_implementation/product-page-extractor").call(
    run_input={
        "productUrls": [
            "https://www.example-store.com/products/classic-canvas-sneaker",
            "https://shop.example.com/p/desk-lamp-black",
        ],
        "includeVariants": True,
        "maxItems": 1000,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item["status"] == "ok":
        print(item["name"], item["price"], item["currency"], item["inStock"], item.get("gtin"))
    else:
        print("skipped", item["url"], item["status"])

Pricing

Pay per event, with no subscription and no platform usage charge on top:

Event	Price
Product extracted	$0.0015 per row with status `ok` ($1.50 per 1,000 products)

There is no start fee. Blocked pages, pages without product data, HTTP errors, robots.txt skips and timeouts are free. Tracking 500 product pages once a day costs about $0.75 a day if every page returns data.

Limits and what it does not do

Sites with strong bot protection often answer datacenter traffic with a block page or let the request time out. Those rows come back as blocked or error and cost nothing. The actor does not try to get around blocks.
Only published structured data is returned. If a store leaves GTIN or stock out of its Schema.org data, those fields are empty.
No browser. Prices that appear only after JavaScript runs, with no structured data behind them, are not seen.
Prices are what a visitor from a US datacenter with no cookies sees. Stores that change price or currency by country may show you different values.
One product per URL. Category and search pages are not crawled. Collect product URLs first, for example from the store's sitemap.

Use the data in line with each source site's terms.

Try it on the Apify Store: https://apify.com/pistachio_implementation/product-page-extractor

How to get Kalshi and Polymarket odds and price history in one spreadsheet

Hay Equipos — Fri, 02 Oct 2026 21:46:20 +0000

Prediction markets put a live price on questions like rate decisions, inflation prints, elections and the weather. If you are building a dashboard, a research notebook or a model, you quickly hit two problems: Kalshi and Polymarket each have their own APIs with their own field names and units, and comparing them means writing and maintaining two clients.

This guide shows how to get both into one normalized table with a small Apify Actor published by Hay Equipos called Prediction Markets Data: Kalshi and Polymarket Odds API. It reads only the exchanges' official public market data APIs. No login, no account and no personal data.

This tool provides data only. It is not betting advice or financial advice.

What the tool returns

Every row is one market with the same fields whichever exchange it came from:

source, market ID, link, event ID and title, the question and outcome label
category and tags, status, result, open and close times
best Yes bid and ask, last price, and impliedProbability
total volume, 24 hour volume, volumeUnit, liquidity and open interest
the outcomes with their prices (and token IDs on Polymarket)
optionally an order book snapshot and a price history series for the Yes side

Prices run from 0 to 1, so 0.62 means the market puts the Yes outcome at about 62%. impliedProbability is the midpoint of the best bid and ask when both exist, otherwise the last price (Kalshi) or the listed outcome price (Polymarket).

A row looks like this (values are illustrative, arrays shortened):

{
  "source": "kalshi",
  "marketId": "KXEXAMPLE-26DEC-T3.0",
  "eventTitle": "Example: inflation above 3.0% in December?",
  "question": "Example: inflation above 3.0% in December?",
  "outcomeLabel": "Yes",
  "category": "Economics",
  "status": "open",
  "closeTime": "2027-01-14T13:30:00Z",
  "yesBid": 0.41,
  "yesAsk": 0.43,
  "lastPrice": 0.42,
  "impliedProbability": 0.42,
  "volume": 185400,
  "volumeUnit": "contracts",
  "openInterest": 52310,
  "orderBook": {
    "side": "yes",
    "bids": [{ "price": 0.41, "size": 1200 }],
    "asks": [{ "price": 0.43, "size": 950 }]
  },
  "priceHistory": [{ "timestamp": "2026-10-01T00:00:00.000Z", "price": 0.39 }]
}

Step by step in the Apify Console

Open the Actor from its Apify Store page and sign in to Apify Console.
In the Input tab, choose the Exchanges: Kalshi, Polymarket or both.
Optionally enter a Keyword (matched against the question, event title or outcome) and Categories such as Politics, Economics, Sports, Crypto or Climate and Weather.
Pick the Market status: open, closed or settled (settled is Kalshi only).
Set Minimum total volume and Maximum markets per exchange. Markets are sorted by total volume, highest first.
Switch on Include order book snapshot and Include price history if you need them, and choose the depth, number of days and hourly or daily spacing.
Click Start, then export from the Output tab as CSV, JSON or Excel.

Use Apify schedules to run it hourly or daily and build your own history.

How to call it from code

With curl:

curl -X POST "https://api.apify.com/v2/acts/pistachio_implementation~prediction-markets-data/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sources": ["kalshi", "polymarket"], "keyword": "fed", "status": "open", "maxMarkets": 25}'

In Python, with the apify-client package:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("pistachio_implementation/prediction-markets-data").call(
    run_input={
        "sources": ["kalshi", "polymarket"],
        "categories": ["Economics", "Economy"],
        "minVolume": 1000,
        "maxMarkets": 50,
        "includePriceHistory": True,
        "historyDays": 30,
        "historyInterval": "1d",
    }
)
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row["source"], row["question"], row["impliedProbability"], row["volumeUnit"])

A full Kalshi scan can take a minute or more, so for larger pulls the Python client, which waits for the run to finish, is the safer choice.

Pricing

Pay per event, with no subscription and no charge for platform usage:

Market record saved: $0.001 ($1 per 1,000 markets)
Order book snapshot added: $0.001 per market
Price history added: $0.001 per market

A daily pull of the top 100 markets on both exchanges with order books comes to about $0.40. You can cap your spend with the maximum charge setting on each run, and the Actor stops cleanly when the cap is reached.

Limits and what it does not do

Kalshi has no server side keyword search, so the Actor scans all listing pages for the chosen status (about 65 pages and over 100,000 markets for open markets). Closed and settled history is very large, so raise Maximum listing pages per exchange to go deeper.
Volume units differ. Kalshi counts contracts and Polymarket counts US dollars. The volumeUnit field says which.
Yes side only for order books and price history (the first outcome on Polymarket). On binary markets the No side is the mirror image.
No multi leg combo markets (parlays) from Kalshi.
No automatic matching across exchanges. Each exchange words its markets differently. Filter both with the same keyword and compare question, closeTime and impliedProbability yourself.
Polymarket is blocked in some countries at the network level. Apify's servers are not affected, but if you run the Actor on your own machine in a blocked country, the Polymarket part fails while Kalshi results are still saved.
No trading. It reads public market data. It cannot place orders and needs no account.
Data is as published by each exchange at the time of the run.

The Actor is not affiliated with Kalshi or Polymarket. Please use the data in line with each exchange's terms.

Try it on the Apify Store: https://apify.com/pistachio_implementation/prediction-markets-data

How to run an on page SEO audit for a list of URLs

Hay Equipos — Fri, 02 Oct 2026 21:45:45 +0000

Checking titles, meta descriptions, headings, canonicals and noindex tags page by page in a browser is fine for one landing page. For a site with two hundred pages, or a client list of competitor pages, you want one table: every page, a score, and a list of what to fix.

On Page SEO Audit by Hay Equipos does that. Give it a list of URLs, or one start page and let it crawl the site, and it returns one row per page with a score, every issue it found (error, warning or notice) and a fix hint for each.

What it checks

Title, meta description, H1 and heading structure, canonical, indexability (noindex, the X-Robots-Tag header and robots.txt), hreflang, language, mobile viewport, Open Graph and Twitter cards, structured data (JSON-LD and microdata), image alt text, thin content, text to HTML ratio, server response time, page weight, redirects, HTTPS and mixed content, favicon, empty links and, optionally, broken links.

What you get back

One row per page. Trimmed example with illustrative values:

{
  "url": "https://www.example.com/pricing",
  "finalUrl": "https://www.example.com/pricing",
  "success": true,
  "statusCode": 200,
  "responseTimeMs": 380,
  "htmlBytes": 48210,
  "score": 71,
  "errorCount": 1,
  "warningCount": 3,
  "noticeCount": 2,
  "title": "Pricing",
  "titleLength": 7,
  "metaDescription": null,
  "h1Count": 2,
  "canonical": "https://www.example.com/pricing",
  "isIndexable": true,
  "hasViewport": true,
  "wordCount": 412,
  "imagesMissingAlt": 3,
  "issues": [
    { "severity": "warning", "code": "title_too_short", "message": "The title is only 7 characters.", "fix": "Use 30 to 60 characters with the main keyword first." },
    { "severity": "warning", "code": "description_missing", "message": "The page has no meta description.", "fix": "Add a meta description of 70 to 160 characters that summarizes the page." }
  ],
  "auditedAt": "2026-09-27T06:14:02.518Z"
}

Every row also has fields such as metaRobots, xRobotsTag, hreflang, openGraph, twitterCard, structuredDataTypes, textToHtmlRatio, internalLinkCount, externalLinkCount, mixedContentCount and hasFavicon. With the link check on, you also get linksChecked and brokenLinks.

How the score works: each page starts at 100 and loses 12 points per error, 5 per warning and 1 per notice, with a floor of 0. Broken links count as one error per page.

At the end of each run, a site summary is saved in the run's key value store under SUMMARY: pages audited, average score, how often each issue appeared, duplicate titles and duplicate meta descriptions across the pages audited.

Step by step in the Apify Console

Open the actor on the Apify Store (link at the end) and click Try for free.
Add your pages to Page URLs. To audit a whole site, add the home page and turn on Crawl the site. The crawl follows links on the same site (www and non www count as one) and skips images, PDFs, scripts and other files.
Set Maximum pages (default 20, up to 2,000 per run).
Optionally turn on Check for broken links, plus Also check external links, and set Links to check per page (default 50).
Keep Respect robots.txt on unless the site is your own. Skipped pages are free.
Set Parallel requests (default 3). Requests to any one site are always spaced one second apart.
Click Start. Use the "Audit by page" table view, or export to CSV or Excel. Open the key value store to read SUMMARY.

To monitor a site weekly, save the input as a task and add a schedule.

Calling it from code

With curl (the synchronous endpoint suits small audits; use the Python client for big crawls):

curl -X POST \
  "https://api.apify.com/v2/acts/pistachio_implementation~on-page-seo-audit/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://www.example.com/", "https://www.example.com/pricing"],
    "crawlSite": false,
    "checkLinks": true
  }'

With Python and the apify-client package:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("pistachio_implementation/on-page-seo-audit").call(
    run_input={
        "urls": ["https://www.example.com/"],
        "crawlSite": True,
        "maxPages": 100,
        "checkLinks": True,
    }
)

for page in client.dataset(run["defaultDatasetId"]).iterate_items():
    errors = [i["code"] for i in page.get("issues", []) if i["severity"] == "error"]
    print(page.get("score"), page["url"], errors)

summary = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("SUMMARY")["value"]
print("Average score:", summary["averageScore"])
print("Duplicate titles:", len(summary["duplicateTitles"]))

Every issue has a stable code, a severity, a readable message and a fix, so you can route them into a ticketing system or an AI agent without parsing text.

Pricing

Pay per event, with no subscription and no platform usage charge on top:

Event	Price
Actor start (Apify's standard start event)	$0.00005 per run
Page audited	$0.003 per page ($3 per 1,000 pages)

Only pages that answered with a status below 400 and were not a bot check are charged. Unreachable pages, bot checks, non HTML files and robots.txt skips are free. The broken link check is included in the page price, though it makes runs slower. You can set a maximum charge per run and the actor stops cleanly when it is reached.

Limits and what it does not do

Plain HTTP only. It audits the HTML the server sends. Content added later by JavaScript is not seen.
No Core Web Vitals or lab speed scores. It reports server response time and page weight, not browser rendering metrics. Use a browser based tool for those.
Sites behind a bot check return a free error row instead of a misleading audit.
One request per second per site, so a 500 page crawl takes about 8 to 9 minutes.
The crawl stays on the start site, and a run audits at most 2,000 pages.
Duplicate detection covers only the pages audited in that run.

Audit sites you own or have permission to test, and use the results in line with each site's terms.

Try it on the Apify Store: https://apify.com/pistachio_implementation/on-page-seo-audit

How to clean an email list and remove invalid addresses before sending

Hay Equipos — Fri, 02 Oct 2026 21:39:58 +0000

Every bounced email costs you a little sender reputation, and a list exported from a form, a CRM or an old spreadsheet is full of addresses that will bounce: typos like gmial.com or .con, domains that no longer exist, throwaway inboxes and shared info@ addresses. You want to find those before you press send, not after.

This guide shows how to check a whole list in one run with a small Apify Actor published by Hay Equipos called Email Validator: Syntax, MX, Disposable and Role Check. It works from public DNS only. It never sends an email and never connects to anyone's mail server.

What the tool checks

For every address you get one row that tells you:

whether the syntax is valid
whether the domain exists and accepts email (MX records, and the "null MX" record a domain uses to say it takes no mail)
which mail provider runs it (Google Workspace, Microsoft 365, Proton, Zoho and others)
whether it is a disposable provider (a list of about 9,000 domains)
whether it is a role inbox such as info, sales, support or noreply (about 90 role names)
whether it is free webmail such as Gmail or Yahoo
a suggested fix when the domain looks like a typo
optionally, the domain's SPF record and DMARC policy

Each row ends in a verdict and a plain sentence reason:

deliverable_domain: valid syntax, the domain accepts email, not disposable, not a role inbox.
risky: the domain works, but the address is disposable, a role inbox, has no MX record, or the domain's name servers fail.
invalid: bad syntax, the domain does not exist, or it publishes a null MX.
unknown: the DNS lookup failed on the Actor's side. These rows are free; run them again.

Two sample rows (values are illustrative):

[
  {
    "input": "jane@example.con",
    "email": "jane@example.con",
    "verdict": "invalid",
    "reason": "The domain does not exist",
    "isValidSyntax": true,
    "domainExists": false,
    "hasMx": false,
    "isDisposable": false,
    "isRoleAccount": false,
    "suggestedEmail": "jane@example.com"
  },
  {
    "input": "sales@example.org",
    "verdict": "risky",
    "reason": "Role account (a shared inbox such as info@ or sales@)",
    "domainExists": true,
    "hasMx": true,
    "mxRecords": ["1 aspmx.l.google.com"],
    "mailProvider": "Google Workspace",
    "isRoleAccount": true,
    "isFreeProvider": false
  }
]

A summary with the count per verdict is saved in the run's key value store as SUMMARY.

Step by step in the Apify Console

Open the Actor from its Apify Store page and sign in to Apify Console.
In the Input tab, add addresses to Emails, one per line, or paste a whole spreadsheet column into Or paste a list (new lines, commas and semicolons all work).
Keep Check the domain in DNS on. Turn it off only if you want a syntax only check.
Switch on Also return SPF and DMARC if you want those fields. It costs nothing extra.
Leave Skip duplicates on so each address is checked once.
Click Start, then export the results from the Output tab as CSV, JSON or Excel. Filter on verdict and keep the deliverable_domain rows.

How to call it from code

With curl, using your token from an environment variable:

curl -X POST "https://api.apify.com/v2/acts/pistachio_implementation~email-validator-mx-check/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"emails": ["jane@example.com", "info@example.org"], "checkSpfDmarc": true}'

In Python, with the apify-client package:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("pistachio_implementation/email-validator-mx-check").call(
    run_input={"emails": ["jane@example.com", "info@example.org"]}
)
keep = [
    row["input"]
    for row in client.dataset(run["defaultDatasetId"]).iterate_items()
    if row.get("verdict") == "deliverable_domain"
]
print(keep)

For large lists, prefer the Python client: the synchronous curl endpoint is meant for runs that finish within a few minutes.

Pricing

Pay per event: $0.0005 per email checked, which is $0.50 per 1,000 emails. There is no start fee and no platform usage charge on top. Rows with the verdict unknown are not charged, and SPF and DMARC come at no extra cost. If you turn DNS checks off, each row gets valid_syntax or invalid and is charged at the same price. You can cap the spend of any run with the maximum charge setting in Apify.

Limits and what it does not do

No mailbox probing. The Actor does not connect to mail servers to ask whether a specific mailbox exists (SMTP verification). It cannot tell you that jane@company.com exists while jnae@company.com does not.
Catch all domains are not detected, because that also needs SMTP.
A deliverable_domain verdict means the domain can receive mail, not that the person will.
The disposable list is a snapshot of a public blocklist maintained on GitHub, updated with each release of the Actor, so a brand new throwaway service may not be on it yet.
Up to 100,000 addresses per run, at roughly 1,000 addresses a minute. Lists with many repeated domains go faster because each domain is looked up once.

What it does remove is everything that is certain to fail or likely to hurt you: broken syntax, dead domains, domains without mail, disposable and role addresses, and typos. If you need mailbox level certainty, run this first and send only the deliverable_domain rows to an SMTP verifier.

Your list stays in your own Apify storage. Only the domain part of each address is looked up in public DNS.

Try it on the Apify Store: https://apify.com/pistachio_implementation/email-validator-mx-check

How to check domain expiry dates, DNS records and SSL certificates in bulk

Hay Equipos — Fri, 02 Oct 2026 21:39:23 +0000

If you manage more than a handful of domains, you already know the problem. A domain renewal slips, a certificate expires on a Sunday, or a client asks whether their DMARC policy is set, and the answer lives in three different places: a WHOIS lookup, a DNS tool and a browser padlock. Checking fifty domains that way takes an afternoon.

This guide shows how to get all three answers for a whole list of domains at once, as a spreadsheet, using a small Apify Actor published by Hay Equipos called Domain WHOIS, DNS and SSL Expiry Lookup (RDAP).

What the tool returns

You give it domains or full URLs. For each one you get a single row with:

Registration data from RDAP (the structured replacement for WHOIS): registrar, registrar IANA ID, abuse contact, created, updated and expiry dates, domain age in days, days until expiry, status codes, name servers and DNSSEC.
DNS records: A, AAAA, MX, NS, TXT, CAA, plus the SPF record and the DMARC policy.
The SSL certificate from a TLS handshake on port 443: issuer, subject, valid from and to, days left, whether the chain is trusted (with the error code if not) and the names the certificate covers.

Here is what a row looks like. The values are illustrative:

{
  "input": "https://www.example.com/pricing",
  "domain": "example.com",
  "host": "www.example.com",
  "success": true,
  "rdapStatus": "ok",
  "isRegistered": true,
  "registrar": "Example Registrar, Inc.",
  "createdAt": "2012-03-14T10:22:05.000Z",
  "expiresAt": "2027-03-14T10:22:05.000Z",
  "domainAgeDays": 5315,
  "daysUntilExpiry": 163,
  "nameservers": ["ns1.example-dns.net", "ns2.example-dns.net"],
  "mxRecords": ["10 mx1.example-mail.com"],
  "dmarcPolicy": "quarantine",
  "hasWebsite": true,
  "acceptsEmail": true,
  "sslStatus": "ok",
  "sslValid": true,
  "sslIssuer": "Let's Encrypt",
  "sslValidTo": "2026-11-20T08:00:00.000Z",
  "sslDaysUntilExpiry": 49
}

URLs and subdomains are fine. The registrable domain is worked out with the public suffix list, so https://www.bbc.co.uk/news is looked up as bbc.co.uk for registration and DNS, while the SSL check uses the exact host www.bbc.co.uk.

A domain that is not registered comes back with rdapStatus: "not_found" and isRegistered: false, so the same run doubles as a bulk availability check.

Step by step in the Apify Console

Open the Actor from its Apify Store page and sign in to Apify Console (a free account works).
In the Input tab, paste your domains into Domains or URLs, one per line. You can also paste a list separated by spaces, commas or new lines into Or paste a list, for example a column copied from a spreadsheet.
Leave WHOIS (RDAP), DNS records and SSL certificate switched on, or turn off any check you do not need.
Click Start.
When the run finishes, open the Output tab and export the dataset as CSV, JSON or Excel.

For renewal monitoring, schedule the Actor to run weekly with your domain list, then filter the results on daysUntilExpiry or sslDaysUntilExpiry.

How to call it from code

With curl, the synchronous endpoint starts a run and returns the rows when it finishes. Keep your token in an environment variable:

curl -X POST "https://api.apify.com/v2/acts/pistachio_implementation~domain-whois-dns-ssl-lookup/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains": ["example.com", "example.org"], "whois": true, "dns": true, "ssl": true}'

In Python, with the apify-client package (pip install apify-client):

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("pistachio_implementation/domain-whois-dns-ssl-lookup").call(
    run_input={"domains": ["example.com", "example.org"]}
)
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row["domain"], row.get("daysUntilExpiry"), row.get("sslDaysUntilExpiry"))

The synchronous curl endpoint is best for short lists. For thousands of domains, use the Python client, which waits for the run to finish however long it takes.

Pricing

The Actor uses pay per event pricing: $0.0015 per domain looked up, which is $1.50 per 1,000 domains, with registration, DNS and SSL checks all included. There is no start fee and no platform usage charge on top. A domain is charged when at least one source returned facts about it. Invalid input and domains where every source failed come back as free error rows. You can cap the spend of any run with the maximum charge setting in Apify.

Limits and what it does not do

No registrant data. Owner names, emails, phone numbers and addresses are never collected, even where a registry still shows them. You get registrar, dates, status and name servers only.
Some country registries have no RDAP service, for example .de. For those, rdapStatus is no_rdap_server and the registration fields are empty, while DNS and SSL still work and the row is still one charge.
Registries rate limit RDAP. The Actor keeps to about one request every 0.7 seconds per registry, so 1,000 .com domains take roughly 12 to 25 minutes. Lists that mix many endings run faster.
The SSL check only looks at port 443 of the host you give. Sites without HTTPS show sslStatus: "error".
Some registries do not publish a creation date, so domainAgeDays can be empty. The Actor does not guess.
Up to 20,000 domains per run.
It does not scrape any website, and it does not tell you who owns a domain.

Registries and DNS providers publish this data under their own terms, so use the results in line with those terms.

Try it on the Apify Store: https://apify.com/pistachio_implementation/domain-whois-dns-ssl-lookup