DEV Community: eden

What 80 E-Commerce Product Pages Reveal About Structured Data for AI Crawlers (Shopify & WooCommerce)

eden — Fri, 30 Oct 2026 15:58:12 +0000

When shoppers ask AI assistants about products—such as price, stock availability, shipping costs, and return windows—those systems rely on a mix of web search indices, page text, and structured data (application/ld+json).

To see what public technical signals stores actually expose today, we ran an honest HTTP audit across a chosen sample of 100 online stores (99 scored, 1 blocked by firewall, 80 with a readable product page).

Key Technical Findings

Native Platform Files (/llms.txt and /.well-known/ucp):
Shopify now serves /llms.txt and /.well-known/ucp natively across stores. WooCommerce and custom stores do not generate these automatically unless configured.
The Product Schema Gap (shippingDetails & hasMerchantReturnPolicy):
While most themes output basic Offer fields (price, priceCurrency, availability), policy markup inside Product JSON-LD was almost always missing in raw HTML:
78 of 80 product pages had no shippingDetails in their structured data.
76 of 80 product pages had no hasMerchantReturnPolicy in their structured data.

Important qualifier: This is a technical check of public structured data on a non-random sample we selected. A missing schema field does not mean a store lacks a shipping or return policy on its website, and structured data alone does not guarantee or predict what any AI assistant will recommend.

Try the Free Read-Only Check

I built CitedCart so Shopify and WooCommerce store owners can inspect their own public product and policy markup (read-only, no store login required):

Run a free store check: https://citedcart.com
View a full sample report (no email required): https://citedcart.com/sample-report.html?lang=en

Who's hiring the most in tech this week: SpaceX, Wise, ElevenLabs and OpenAI lead (Oct 15, 2026)

eden — Thu, 15 Oct 2026 07:39:38 +0000

Data pulled on October 15, 2026 (05:22 UTC) straight from the public job boards of 78 well-known tech companies (Greenhouse, Lever, Ashby and others). Together they list 16,045 open jobs, and 1,285 of them were posted or updated in the last 7 days.

Who posted the most jobs this week

#	Company	New/updated jobs (7 days)	Open jobs	Top department	Remote
1	SpaceX	141	2745	Starlink Enterprise Sales & Account Management	17
2	Wise	95	407	Engineering	1
3	ElevenLabs	56	147	Engineering & Product	129
4	OpenAI	53	804	Go To Market	527
5	Stripe	48	728	Account Executives (EMEA)	114
6	Snowflake	45	357	Engineering	172
7	Databricks	44	891	Engineering - Pipeline	71
8	MongoDB	42	401	Sales Growth EMEA	0
9	Toast	38	313	Sales : Retail Sales AEs	52
10	Cloudflare	37	445	Solution Engineering	3
11	Anthropic	35	647	Sales	59
12	Harvey	35	344	Go to Market	275
13	Affirm	31	206	Consumer Engineering	199
14	Elastic	30	382	Platform - Elasticsearch	0
15	GitLab	25	215	Platforms Engineering	192
16	Datadog	24	438	Enterprise Sales	56
17	Samsara	24	257	Customer Success	225
18	Twilio	24	141	Engineering	141
19	Roblox	23	260	Software Engineering	8
20	Airbnb	23	164	Software Engineering	19
21	Verkada	22	315	Solutions Engineering	0
22	Waymo	19	364	Pipeline (N/A)	10
23	Flexport	19	206	Sales	4
24	Zoox	17	239	Software	0
25	Pinterest	17	156	EPD Ads and Ads Product	64

Biggest job boards right now

#	Company	Open jobs	ATS
1	SpaceX	2745	greenhouse
2	Databricks	891	greenhouse
3	OpenAI	804	ashby
4	Stripe	728	greenhouse
5	Anthropic	647	greenhouse
6	Cloudflare	445	greenhouse
7	Datadog	438	greenhouse
8	Wise	407	smartrecruiters
9	MongoDB	401	greenhouse
10	Elastic	382	greenhouse
11	Waymo	364	greenhouse
12	Snowflake	357	ashby
13	Harvey	344	ashby
14	Verkada	315	greenhouse
15	Palantir	314	lever

How this was made

Every number comes from the company's own public job board API, fetched at the time above. "New/updated" uses the posting date each board publishes (some boards report the last update instead of the first publish date). Companies on Workday or custom career sites are not included.

The data was collected with the same code as the ATS Job Scraper (every open job from a company's Greenhouse, Lever, Ashby, Workable, Recruitee or SmartRecruiters board) and the Company Intent Radar (hiring + Google ads + SEO signals per domain), both on the Apify Store. You can run the same lookup for your own list of companies.

The report also lives at https://edtool79-collab.github.io/hirepulse-tools/hiring-report.html and is updated every week.

Disclosure: I built these tools. This report is generated automatically every week.

See every Google ad a competitor runs, from just their domain (cURL, Python, n8n)

eden — Thu, 15 Oct 2026 03:58:47 +0000

Google's Ads Transparency Center lists the ads that advertisers run on Google Search, YouTube, Shopping, Play and Maps, with the dates each ad was first and last shown. For media buyers, agencies and B2B sales teams, that is a goldmine: what are my competitors running right now, since when, and in which format?

Getting it into a pipeline is the annoying part. Some tools ask you to open the Transparency Center, find the advertiser by hand and paste its URL. That is fine for one brand. It is not fine for 50 competitors or a list of 1,000 prospects.

So I built a small tool that takes plain domains (nike.com, adidas.com) and returns the ads as JSON.

What it does

The Google Ads Transparency Scraper runs on Apify:

Domains in, ads out. No advertiser IDs, no copy-pasting Google URLs.
No browser, no proxies. It reads the data through an API backend, so runs take seconds.
Two output modes: ads returns one record per ad, for swipe files and monitoring. summary returns one compact record per domain (does it advertise, how many ads, which formats, latest ad), for lead lists, Clay tables and AI agents.
"Only new ads" monitoring: schedule it daily or weekly with only_new_ads: true, and each run returns only ads you have not seen before.
Every advertiser behind a domain: results include every advertiser whose ads lead to the domain, not only the brand itself. When I looked up adidas.com, the advertisers included "Adidas America, Inc", the media agency "MediaCom L.L.C" and an individual advertiser name I did not recognize. That is useful for brand protection too.

1. Quick start with cURL

curl -X POST "https://api.apify.com/v2/acts/hirepulse~google-ads-transparency-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "domains": ["nike.com"], "country": "US", "max_ads_per_domain": 5 }'

2. Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("hirepulse/google-ads-transparency-scraper").call(run_input={
    "domains": ["gusto.com", "rippling.com"],
    "country": "US",
    "platform": "google_search",   # all, google_search, youtube, google_shopping, google_play, google_maps
    "format": "all",               # all, text, image, video
    "max_ads_per_domain": 40,      # up to 120
    "mode": "ads",
})

for ad in client.dataset(run["defaultDatasetId"]).iterate_items():
    if ad.get("ad_id"):
        print(ad["domain"], ad["advertiser_name"], ad["format"], ad["days_running"], "days", ad["transparency_url"])

3. n8n: weekly "new competitor ads" alert

Schedule Trigger: every Monday at 09:00.
HTTP Request (POST) to the run-sync-get-dataset-items URL above, with:

   { "domains": ["competitor1.com", "competitor2.com"], "only_new_ads": true, "max_ads_per_domain": 40 }

IF ad_id is not empty (records without ad_id are notes, such as "No new ads since the last run").
Slack or email: post the advertiser, format and transparency_url of each new ad.

What the output looks like

Ads mode (a real record from a test run on 14.10.2026):

{
  "domain": "nike.com",
  "advertiser_name": "Nike, Inc.",
  "advertiser_id": "AR16735076323512287233",
  "advertiser_verified": true,
  "ad_id": "CR05432776185081757697",
  "format": "text",
  "first_shown": "2023-11-16T21:51:37Z",
  "last_shown": "2026-10-14T15:10:53Z",
  "days_running": 1062,
  "last_shown_days_ago": 0,
  "preview_image_url": "https://tpc.googlesyndication.com/archive/simgad/8151844300521207182",
  "preview_image_width": 348,
  "preview_image_height": 557,
  "transparency_url": "https://adstransparency.google.com/advertiser/AR16735076323512287233/creative/CR05432776185081757697?region=US",
  "country": "US"
}

Yes, that text ad has been running for almost three years.

Summary mode (real, adidas.com, all countries):

{
  "domain": "adidas.com",
  "runs_google_ads": true,
  "total_ads": 6000,
  "ads_sampled": 40,
  "text_ads": 33,
  "image_ads": 3,
  "video_ads": 4,
  "latest_ad_shown": "2026-10-14T15:29:10Z"
}

Pricing

Pay per use, no subscription:

$0.005 per domain checked. Also when the domain runs no ads, because "this company does not advertise on Google" is an answer too.
$0.0015 per ad returned ($1.50 per 1,000 ads). Summary mode is charged per domain only.
Higher Apify plans get volume discounts.
Free Apify plan: try one domain with up to 5 ads.

Job	Mode	Cost
1 competitor, 40 ads	ads	$0.065
5 competitors, 200 ads	ads	$0.325
1,000 prospect domains	summary	$5.00

Honest limitations

No spend or impression numbers. Google does not publish them for commercial ads, so no tool can give you real numbers.
No separate ad text field. Google renders text ads as images. You get the image (preview_image_url), which shows the full ad copy.
Videos: you get Google's preview link, not an MP4 file.
Up to 120 recent ads per domain per run.

Try it

👉 Google Ads Transparency Scraper on Apify

Using an AI agent? Connect it through Apify's MCP server: https://mcp.apify.com/?tools=hirepulse/google-ads-transparency-scraper

Disclosure: I built this tool.

Hiring signals in Clay: which of your target accounts are hiring right now

eden — Wed, 14 Oct 2026 17:24:34 +0000

A company that is hiring for a role is usually about to buy what that role needs. A new "Head of RevOps" means a CRM project. Five open "Data Engineer" roles mean a data stack budget. That is why hiring is one of the most used buying signals in outbound.

Clay can run any Apify Actor as an enrichment, so you can add live job data to every row of an account list. This post shows two setups with two small pay-per-use Actors I built:

Your accounts are known. Get every open job, plus what changed since last week, from each company's own careers page.
Your accounts are not known yet. Search Google Jobs for a role in a city and get the companies hiring for it.

Setup 1: open jobs for each company in your table

The ATS Job Scraper takes a company domain, finds its job board (Greenhouse, Lever, Ashby, Workable, Recruitee, SmartRecruiters) and returns a one-row summary:

{
  "record_type": "company_summary",
  "company_domain": "linear.app",
  "total_active_jobs": 31,
  "new_jobs": 2,
  "new_job_titles": ["Senior Product Engineer", "Enterprise Account Executive"],
  "jobs_by_department": { "GTM": 15, "Product": 13, "Operations": 3 },
  "top_locations": { "North America": 19, "Europe": 8 },
  "top_technologies": { "React": 8, "TypeScript": 8 }
}

(Example of the output format.)

In Clay:

In your account table, click Add enrichment, search for Apify and pick Run Apify Actor.
Connect your Apify account (API token from the Apify console, Integrations page).
Choose the Actor hirepulse/ats-job-pulse.
Enter the input as JSON and insert your domain column as the value:

{
  "companies": ["<your Domain column>"],
  "output_mode": "summary_only"
}

Map total_active_jobs, jobs_by_department and new_job_titles into columns.

Now you can filter: "has 3+ open sales roles", "opened an engineering role this month", "hires in Europe".

Cost: $0.05 per company found. Companies whose job board cannot be found are free. 500 accounts cost at most $25.

Weekly changes: add "track_diff": true and a fixed "state_id". Each run then reports new_jobs and closed_jobs since the previous run with the same state_id, which is the real signal: a company that just started hiring.

Setup 2: find companies hiring for a role

The Google Jobs Scraper searches Google Jobs, which collects listings from LinkedIn, Indeed and company career sites:

{
  "queries": ["revops manager", "salesforce administrator"],
  "country": "US",
  "location": "Austin,Texas",
  "max_jobs_per_query": 100
}

Each job comes back with company, title, location, salary (parsed into salary_min and salary_max when shown), posted_ago and apply_url. When the apply link is on a company job board such as Workday, the full description is included too.

Run it once from the Apify console or from Clay with a single-row table, then import the dataset into Clay (Clay can pull the results of an Apify Actor run as a source). De-duplicate on company, and you have a list of companies that need what you sell, in the city you cover.

Cost: $0.005 per search that returns jobs plus $0.001 per job. 200 jobs cost about $0.21.

Tips

Start with summary_only in Setup 1. One row per company keeps Clay credits and Apify costs low.
Filter by title in the Actor, not in Clay: "title_keywords": ["sales", "account executive"] returns only the jobs you care about.
You can test both on Apify's free plan before you commit. The Google Jobs Actor returns a 10-job preview there; full results need a paid Apify plan.

Limitations

Setup 1 works for companies that use one of the six supported job boards. Companies on Workday, iCIMS or Taleo, or with a custom careers site, come back as not found and are not charged.
Google Jobs results are what Google shows at search time; many listings are reposted on several boards, so de-duplicate by company and title.

Questions or a job board you need? Ask in the comments.

Google Jobs results as JSON, for $1 per 1,000 jobs (no proxies, no browser)

eden — Wed, 14 Oct 2026 16:40:13 +0000

Google Jobs is the best free job index on the web. It pulls listings from LinkedIn, Indeed, company career sites and hundreds of smaller boards into one search. What it does not have is an API.

The scrapers that fill that gap tend to break: they drive a browser through Google, get blocked, and you pay for failed runs. In October 2026 the most used one on the Apify Store failed 19% of its runs and charged $0.35 to $0.90 per 100 jobs, and another succeeded in only 1 of 500 runs in a month.

So I built one that does not scrape Google from a browser at all. It reads Google Jobs through a commercial search-data API, so there are no proxies to rotate and nothing to break. This post shows how to use it.

What you get per job

You send one or more searches and a country, optionally a city. You get one record per job:

{
  "query": "data engineer",
  "country": "US",
  "position": 1,
  "title": "Senior Data Engineer",
  "company": "Acme Corp",
  "location": "Austin, TX",
  "contract_type": "Full-time",
  "salary": "$140K–$180K a year",
  "salary_min": 140000,
  "salary_max": 180000,
  "salary_currency": "USD",
  "salary_period": "year",
  "posted_ago": "2 days ago",
  "source": "LinkedIn",
  "apply_url": "https://..."
}

(Example of the output format.)

What to expect from real data: in a test of 3 US searches (30 jobs), Google showed a salary on 5 of the 30 jobs. Many apply links go straight to the company's own career site. When the apply link is on a company job board (Workday, Greenhouse, Lever or Ashby), the record also includes the full job description; in a test of 79 US jobs that was 16 of them. Other jobs keep the apply_url to the original posting.

Run it

The tool is the Google Jobs Scraper Actor on Apify.

From the console

Open the Actor, type your searches, pick a country, optionally a city such as Austin,Texas, and press Start. Results appear as a table you can export to CSV, Excel or JSON.

From curl

curl -X POST "https://api.apify.com/v2/acts/hirepulse~google-jobs-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["data engineer"], "country": "US", "location": "Austin,Texas", "max_jobs_per_query": 50}'

From Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("hirepulse/google-jobs-scraper").call(run_input={
    "queries": ["account executive", "sales development representative"],
    "country": "GB",
    "employment_types": ["fulltime"],
})
for job in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(job["title"], "|", job["company"], "|", job.get("salary"))

From an AI agent

Add Apify's MCP server with this one tool to Claude, Cursor or any MCP client:

https://mcp.apify.com?tools=hirepulse/google-jobs-scraper

Then ask: "Which companies in Austin are hiring data engineers this week, and what do they pay?"

Pricing

$0.005 per search that returns jobs. Searches with no results are free.
$0.001 per job.
100 jobs from one search cost about $0.11.

You need an Apify account. You can try it on the free Apify plan: each free run returns the first 10 jobs of your first search. Full results need a paid Apify plan, because every search has a real data cost.

If you track specific companies rather than roles, the ATS Job Scraper reads every open job, with full descriptions, straight from a company's Greenhouse, Lever, Ashby or Workable board, and tells you what changed since the last run.

DEV Community: eden

What 80 E-Commerce Product Pages Reveal About Structured Data for AI Crawlers (Shopify & WooCommerce)

Key Technical Findings

Try the Free Read-Only Check

Who's hiring the most in tech this week: SpaceX, Wise, ElevenLabs and OpenAI lead (Oct 15, 2026)

Who posted the most jobs this week

Biggest job boards right now

How this was made

See every Google ad a competitor runs, from just their domain (cURL, Python, n8n)

What it does

1. Quick start with cURL

2. Python

3. n8n: weekly "new competitor ads" alert

What the output looks like

Pricing

Honest limitations

Try it

Hiring signals in Clay: which of your target accounts are hiring right now

Setup 1: open jobs for each company in your table

Setup 2: find companies hiring for a role

Tips

Limitations

Google Jobs results as JSON, for $1 per 1,000 jobs (no proxies, no browser)

What you get per job

Run it

From the console

From curl

From Python

From an AI agent

Pricing

Related