DEV Community: Integration Yard

Catch Google Merchant Center Disapprovals Before Google Does

Integration Yard — Thu, 22 Oct 2026 17:08:51 +0000

TL;DR

Most "Mismatched value (page crawl) [price]" and "Mismatched availability" disapprovals in Google Merchant Center come from drift: your product feed and your live storefront briefly disagree, and Googlebot crawls at that moment.

Drift risk is everywhere in real catalogs. In four public Shopify stores we probed on Oct 20, 2026, 15–85% of variants were on sale (compare-at price set) and 13–82% were sold out. One brand's 250 products expanded to 2,720 variants, each a separate feed item.

Google says it re-crawls disapproved products over "the next few hours or days" and may take up to 12 hours to review a website check, so every drift costs you visibility for a while.

Do the free manual audit below once. Then automate a daily storefront-vs-feed diff. With DriftGuard, 500 variants with no new drift cost about $0.10 a check.

Here's how it usually goes. You change a price in Shopify, start a sale, or sell out of a size. A few days later Merchant Center shows "Mismatched value (page crawl) [price]" or "Mismatched availability". The product is disapproved, Shopping stops serving it, and if it happens often enough, the account gets a warning about inaccurate prices.

Google's fix guidance always comes down to one rule: the price and availability in your feed must match your landing page, your structured data and your checkout. The rule is simple. The hard part is noticing that the feed and the store have drifted apart before Googlebot does.

This guide covers why drift happens, what it costs you, a manual audit you can do today, and how to automate the check. It also includes real data from public Shopify stores on how much of a typical catalog is exposed to drift at any given moment.

Disclosure: we built DriftGuard, the actor used in part 4. Parts 1–3 are useful with or without it.

What Google actually checks
Why feeds drift from the storefront
How exposed is a real catalog? Data from four stores
The manual audit (do this once, today)
Who this protects, and what it's worth
Automate it: storefront vs feed, every day
Get alerted
What it costs
Caveats
FAQ
Try it

What Google actually checks

Google's product data specification says to "accurately submit the product's price and currency, and match with the price from your landing page, structured data, and at checkout." The same applies to sale prices and availability. When they don't match:

Google's crawler visits the landing page and reads the visible price and the structured data (JSON-LD or Microdata).
If those disagree with your feed, the item gets a mismatched value issue and is disapproved.
After you fix it, Google says it re-crawls the product "over the next few hours or days" (Mismatched product price help page). A requested website check can take "up to 12 hours", and you can't submit a new request during that window.
If mismatches are systematic, policy reviewers can flag the account for inaccurate prices (Google help: inconsistency between feed and landing page).

So every drift costs you some hours of visibility per product. Repeated drift puts the account at risk. Google's own list of common causes includes sales that start or end without a feed update, structured data that disagrees with the page, multiple prices on the page, and prices that change by location, cookie or device.

Why feeds drift from the storefront

Cause	What happens
Sync lag	Your feed app pushes updates on its own schedule. Price changes can take hours or days to reach Merchant Center, and deleted variants can linger for weeks.
Sales and compare-at prices	The storefront shows the sale price, but the feed still has the regular price, or the sale's effective-date window is in the wrong timezone.
Sold-out variants	The feed says in stock, but the variant sold out an hour ago. "Continue selling when out of stock" may be handled differently by the feed and the theme.
Price-test and B2B apps	A/B pricing apps, wholesale apps and currency converters change what visitors (and Googlebot) see without touching the feed.
Shopify Markets	There's a feed per country and prices per market, and one of them doesn't update.
Deleted or renamed products	The product is still in the feed but gone from the store (404, then disapproval), or a new product never made it into the feed at all.

Google's automatic item updates soften minor lag by updating price and availability from the crawl. Google is clear, though, that they're not a fix for systematic mismatches. If too many items mismatch, you need to fix the source.

How exposed is a real catalog? Data from four stores

Drift can only happen on fields that change. To see how much of a real catalog is in a changeable state at any moment, we read the public storefront catalogs of four well-known Shopify brands on Oct 20, 2026 (first 250 products each). We've anonymized them by category.

Store	Products	Variants	On sale (compare-at > price)	Sold out
Footwear brand	250	2,720	1,496 (55%)	2,238 (82%)
Bedding brand	250	3,005	510 (17%)	442 (15%)
Cosmetics brand	245	695	105 (15%)	145 (21%)
Watch brand	250	250	212 (85%)	32 (13%)

What this means for your feed:

Sale prices are the norm, not the exception. Every on-sale variant needs price and sale_price in the feed, plus a correct sale_price_effective_date if the sale is scheduled. When a sale starts or ends, every one of those variants has to change in the feed at the same moment it changes on the site.
Stock flips are constant. A footwear catalog where 82% of size/colour variants are sold out has thousands of availability values that can go stale.
Variants multiply the work. Feeds list each size and colour separately:

A manual spot check of "a few products" covers a tiny fraction of a catalog with 10–12 variants per product. That's why disapprovals feel random. The ones you checked were fine.

The manual audit (do this once, today)

In Merchant Center, go to Products → Needs attention. Filter for Mismatched price and Mismatched availability and open View samples.
Open a sample product's attributes and compare the submitted values with what Google collected from the page. For variant products, check that the landing URL includes the variant id, so it preselects the advertised variant.
Open the landing page and confirm that the visible price and stock match the feed.
Run the URL through Google's Rich Results Test. The Offer price and availability in your JSON-LD should match too.
Fix the root cause (sync schedule, sale timing, theme schema, app conflict), re-fetch the feed, and request a website check if the products were flagged.
For bulk issues, download the affected-products CSV from Needs attention and cross-reference it with your feed, as Google's help page suggests.

That works for 10 products. With 500+ variants and daily price changes, you need to automate step 3.

Who this protects, and what it's worth

Shopify merchants: keep best-sellers live on Google Shopping and free listings instead of losing them for hours or days to a re-crawl.
Paid-search and Shopping teams: stop wasting budget on campaigns whose top products are quietly disapproved, and protect the account from inaccurate-price warnings.
Agencies: run one daily check per client feed and send clients a drift report before they notice a drop.
Feed and catalog managers: see feed-only (stale) and storefront-only (missing) items, the silent catalog shrinkage that never shows up as an error.

Automate it: storefront vs feed, every day

DriftGuard does step 3 at scale. On each run it:

Reads your live storefront as any visitor sees it. No login, no API keys, no app install.
Downloads the feed you submit to Google, as RSS/XML or TSV (auto-detected).
Matches every feed item to the right variant with multi-signal product matching, so renamed products, missing SKUs and multi-country feed ids still line up.
Reports every difference with a severity and a fix hint.

Drift	Severity	Why it matters
`price_mismatch`	high	The most common disapproval cause. Active sale-price windows are respected.
`availability_mismatch`	high	The feed says in stock but the storefront is sold out, or vice versa.
`feed_only`	medium	In the feed but gone from the store: a stale listing.
`storefront_only`	medium	A live product missing from the feed: invisible on Shopping.
`title_mismatch`	low	The feed title diverges sharply from the product title.
`attribute_gap`	free section	Missing GTIN/MPN, brand or image.

Setup

Find your feed URL. This is the URL your feed app (Simprosys, DataFeedWatch, Mulwi, etc.) publishes and Merchant Center fetches. It's in the feed app's settings, or in Merchant Center under your data source's fetch URL.
Run it once:

{
  "storeUrl": "https://your-store.myshopify.com",
  "feedUrl": "https://feeds.your-feed-app.com/your-store/google.xml",
  "storeCurrency": "USD",
  "maxProductsPerStore": 500,
  "maxChargeUsd": 1
}

Leave both URLs empty to see a demo run first.

Read the output. Each drift row looks like this:

{
  "driftType": "price_mismatch",
  "severity": "high",
  "title": "Ceramic Mug - 12oz",
  "sku": "MUG-12",
  "feedItemId": "shopify_US_7012345678901_41234567890123",
  "storefrontValue": "16.50",
  "feedValue": "14.00",
  "detail": "Feed shows 14.00 but the storefront charges 16.50.",
  "fixHint": "Update the feed price to match the storefront (or fix the storefront), then re-upload the feed."
}

Schedule it. Save the input as a Task, then create a Schedule. 0 6 * * * (daily) is enough for most stores. Run it more often around sales and launches, which is exactly when the data above says the most fields change.

Get alerted

Simplest: on the task, go to Integrations → Slack or Send email, triggered on Run succeeded.
Only high-severity drifts (Zapier): Apify → Finished Task Run → Apify → Fetch Dataset Items → Filter recordType = drift and severity = high → Slack or email.
Make: Watch Task Runs → Get Dataset Items → filter → Slack.

Repeat drifts are reported every run but billed once, so a daily schedule doesn't keep charging you for the same unfixed issue.

What it costs

Pricing is pay-per-event: $0.001 per run + $0.0002 per variant checked + $0.02 per new drift.

Variants	No new drift	5 new drifts	25 new drifts
500	$0.10	$0.20	$0.60
2,000	$0.40	$0.50	$0.90
5,000	$1.00	$1.10	$1.50

attribute_gap findings and skipped rows are free, and maxChargeUsd caps every run. A daily check of 500 variants with no drift is about $3 a month. Compare that with what one disapproved best-seller costs you in Shopping traffic over the hours to days Google takes to re-crawl it.

Caveats

It compares your storefront with your feed. It doesn't log into Merchant Center, so it catches the drift that would cause disapprovals rather than reading Google's verdict. Theme or JSON-LD problems where the feed and store agree but your schema says something else still need the Rich Results Test.
Public storefronts only. It reads your public storefront and feed, so password-protected stores aren't supported.
Currency. Feed prices in a different currency from storeCurrency are skipped, not flagged. If you use Markets, run one task per country feed.
Large catalogs. Raise maxProductsPerStore. Items past the cutoff are reported as skipped, never as false drifts.
Shopify only for the storefront side. For WooCommerce catalogs, see our competitor price alerts guide.

FAQ

What does "Mismatched value (page crawl) [price]" mean?
Google's crawler saw a different price on your landing page (or in its structured data) than the one in your feed. Fix whichever side is wrong, then request a website check.

Aren't automatic item updates enough?
They help with occasional lag by updating price and availability from the crawl. Google says they're not a substitute for accurate data, and widespread mismatches can still lead to disapprovals or account warnings.

How long does a disapproval last after I fix it?
Google says it re-crawls over "the next few hours or days". A requested website check can take up to 12 hours.

Do I need to install a Shopify app or give API access?
No. DriftGuard reads your public storefront, exactly as any visitor sees it, plus your feed URL.

Which feed formats are supported?
Google Merchant Center RSS/XML and TSV feeds, auto-detected.

How are feed items matched to variants?
With multi-signal product matching that combines several identifiers, so it keeps working when SKUs are missing, titles change or you run feeds for several countries.

Will I be charged every day for the same unfixed problem?
No. A drift is billed once when it first appears and reported, free, on later runs until it's fixed.

Does it work for Meta or TikTok catalogs?
The check works for any feed in Google's RSS/XML or TSV format. Many Meta catalog feeds use the same format, but it's built and tested for Google Merchant Center.

Try it

DriftGuard: Google Merchant Center Feed Price & Stock Checker on Apify. Run the demo with empty inputs, then point it at your store and feed URL and schedule it daily. Questions? Email apify@integrationyard.com.

Data in this post: public storefront catalogs of four Shopify stores (first 250 products each), read-only, Oct 20, 2026. Google Merchant Center help pages checked Oct 20, 2026. Prices are the actor's published pay-per-event prices.

Weibo Hot Search (微博热搜) Data in 2026: Track China's Trending Topics Without Code

Integration Yard — Wed, 21 Oct 2026 17:14:29 +0000

TL;DR

微博热搜 (Weibo Hot Search) is Weibo's live, ranked board of what's trending in China right now — a mix of news, entertainment, product launches and public debate, refreshed continuously.

In a live export on Oct 21, 2026, the top 50 board came back as 50 ranked rows. Heat scores ranged from 113,821 to 893,934 (median 200,505) — the #1 topic carried 7.9x the heat of #50.

Attention is front-loaded: the top 10 topics held 42% of the combined heat across all 50 rows at that moment.

We also pulled the posts behind one trending topic (rank #3 that day). Of 12 merged, deduplicated posts, 11 came from verified accounts, and nearly all the engagement sat on a single post from a major state news account (683 likes, 399 reposts, 75 comments vs. single digits on the rest).

No login, no cookies, no Weibo account needed. $3 per 1,000 rows.

Weibo had 561 million monthly active users and 254 million daily active users as of June 2026 (Weibo Q2 2026 results) — one of the largest public social platforms anywhere, and the default place Chinese-speaking audiences go to see what's happening right now. The 热搜 (hot search) board sits at the center of that: a constantly updating, ranked list of the topics, hashtags and breaking stories pulling the most attention at any given moment.

If you track China-facing brands, follow Chinese public opinion, or build products that need a pulse on what's trending in the Chinese-language internet, 热搜 is usually the first place to look — and historically the hardest to pull into your own tools without manual copy-paste. This guide shows what the hot search board actually looks like in data form, who uses it and why, and how to get it on a schedule. Every number below comes from real runs on Oct 21, 2026.

Disclosure: we build and maintain the Weibo Scraper used here.

Why 微博热搜 matters
The board in one chart: heat is front-loaded
From a trending topic to the posts behind it
Who uses this data
What you get
Scheduling: building a trend history
FAQ
Try it

Why 微博热搜 matters

China's internet population passed 1.125 billion users as of December 2025, at 80.1% penetration (CNNIC 57th Statistical Report). Weibo's hot search board is one of the main places that much attention converges — it's referenced in Chinese entertainment coverage, cited by news outlets as a measure of public interest, and watched closely by brands the moment their name or a competitor's shows up on it.

Because the board updates continuously and ranks everything by a live "heat" score, it works as a real-time signal for:

What's breaking right now — news, product launches, entertainment and sports moments as they happen.
How big something actually got — rank and heat give you a rough but comparable measure of a topic's pull relative to everything else trending that moment.
Where public conversation is heading — a snapshot today, a trend line over weeks of scheduled snapshots.

The board in one chart: heat is front-loaded

One hot_search run returns the full top-50 board: rank, keyword, a heat score, and Weibo's own label (热 "hot", 新 "new", 沸 "surging", or none).

Heat scores on Oct 21, 2026 ranged from 113,821 (rank 50) to 893,934 (rank 1) — a 7.9x spread top to bottom.
Of the 50 rows, 15 carried a "新" (new) label, 6 were "热" (hot), 3 were "沸" (surging), and 26 had no label at that moment.
31 of the 50 were hashtag-style topics (marked isTopic), the rest standalone keywords.

The top 10 ranked topics held 42% of the combined heat across the whole board, and the next 20 held another third. In other words: the board is long, but a handful of topics are doing most of the pulling at any one moment — which is exactly why rank and heat, not just presence on the list, matter for monitoring.

From a trending topic to the posts behind it

A rank and a heat number tell you that something is trending. To see what people are actually posting, you look up the posts behind a specific hot-search topic by keyword. We did this for the #3 topic from the same board snapshot above, a Long March history feature (长征硬核知识点十问十答).

The search returned 12 merged, deduplicated posts from Weibo's search tabs.
11 of the 12 posting accounts were verified — official and institutional accounts dominate the conversation around this kind of topic.
Nearly all the engagement sat on one post from a major state news account: 683 likes, 399 reposts and 75 comments, versus single digits (often zero) on the other 11.

That pattern — a handful of large accounts carrying most of the visible engagement on a given trending topic — is a useful thing to know before you draw conclusions from raw post counts alone.

Who uses this data

Brands and agencies: track mentions of a product, campaign or spokesperson the moment they surface on the board, and catch a complaint or crisis thread early.
Market and media researchers: measure topic momentum over time, compare how different stories or launches performed relative to each other, and build a dataset of what actually trends in China and when.
Analysts and competitive intelligence teams: watch a sector or competitor's share of public attention, benchmarked against everything else trending that day.
AI / LLM teams: fresh, timestamped, Chinese-language trending topics and posts for retrieval-augmented generation, trend summarization or sentiment models that need to know what's current.

What you get

Data	What you get	Typical use
Hot search board (热搜)	Rank, keyword, heat score, label (热/新/沸), whether it's a hashtag topic	real-time trend tracking, topic momentum
Posts by keyword	Text, engagement counts, author and verification status, timestamp, for a specific trending topic or any keyword	seeing what's actually being said about a trend
Post comments	Reactions to a specific post	sentiment on a particular story or statement
User profiles	Follower counts, verification, bio	checking who's driving a topic

No login, no cookies and no Weibo account are needed for any of it. Results export as JSON, CSV or Excel, and there's a compact, token-optimized format for feeding straight into an LLM pipeline.

Scheduling: building a trend history

A single hot-search snapshot is a moment in time. The real value shows up when you schedule it: run the hot search board hourly (or as often as you need) and you build your own history of what trended, when, and how its heat score moved — something a one-off search can't give you. The same applies to tracking posts for a specific keyword or hashtag over the life of a story.

To set it up: save your input as a Task in Apify Console, then attach a Schedule. Send results where you work through Apify's built-in integrations — Google Sheets, Slack or email alerts, webhooks, Make/Zapier/n8n, or directly into an AI agent through the Apify MCP server.

FAQ

Is there an official Weibo API for hot search data?
Weibo's open platform requires a Chinese developer account and an approval process, and access is limited. This gives you the hot search board and related post data right away, with no account needed.

What does "heat" mean on the hot search board?
It's Weibo's own composite score reflecting how much attention a topic is currently pulling — useful for ranking and comparing topics against each other, not a number with an independently documented formula.

How often does the board change?
Continuously. Scheduling hourly runs is the usual way to capture how it moves.

Can I track a specific brand or keyword instead of the whole board?
Yes — search by keyword at any time, whether or not that term is currently on the hot search board.

Do I need a Weibo account or a Chinese phone number?
No. Everything in this guide works without login.

Can I use this with AI agents or LLM pipelines?
Yes. It works through the Apify MCP server, and there's a compact, token-optimized output format built for feeding results into an LLM.

Try it

Weibo Scraper on Apify Store: the input form opens prefilled with a sample run — press Start.
More China data: Bilibili Scraper for China's biggest video community, or the Douban Scraper for movie, TV and book ratings and reviews.
Related guide: Weibo and Bilibili data in 2026.

Get an Alert When a Competitor Changes Prices on Their WooCommerce Store

Integration Yard — Wed, 21 Oct 2026 17:04:57 +0000

TL;DR

Most WooCommerce stores publish their full catalog publicly, including prices, sale prices, SKUs and stock, so a competitor's catalog can be monitored without logins, plugins or page-by-page scraping.

The useful part is diffing: snapshot the catalog on a schedule and send only what changed (new, removed, price changed, stock changed) to Slack, email or Sheets.

In three real stores we checked on Oct 20, 2026, 13–36% of products were out of stock and 1–15% were on sale. Those are the fields that move day to day.

A daily check of a 1,000-product competitor with 10 changes costs about $0.25 a run with our monitor actor. The first (baseline) run is never charged for changes.

Full setup below: the scheduled actor, Zapier, Make, built-in Apify integrations and Python.

If you sell products that other shops also sell, you've probably done this: open a few competitor tabs every Monday, scroll their catalogs, and try to remember what things cost last week. That works for 20 products. It breaks at 200, and it misses the things that matter most: the sale that started Tuesday night, the best-seller that went out of stock on Thursday, the new product line that appeared over the weekend.

Price-monitoring tools exist, but many are built around matching URLs one product at a time, or need a WooCommerce REST API key, which you can only get for your own store. For competitor stores built on WooCommerce, there's a sturdier path: monitor the whole catalog and get told only what changed.

This guide shows that path end to end. Every number in it comes from real public stores or from our actor's published pricing.

Disclosure: we built the WooCommerce Product Feed & Catalog Monitor actor used in this guide.

Why competitor price monitoring matters
What a competitor's catalog tells you
Step 1: snapshot and diff on a schedule
What it costs
Step 2: alerts with Zapier, Make or Apify integrations
Step 3: the code version
Which alerts are actually worth sending
Caveats
FAQ
Try it

Why competitor price monitoring matters

Competitor signals are some of the cheapest market data you can get, and they're perishable:

Price drops on products you both sell decide whether you lose the click on Google Shopping, the marketplace comparison, or the customer who checks two tabs.
Stock-outs are the most underused signal. When a competitor sells out of a popular item, demand moves to whoever still has it. That's the moment to hold your price, not cut it, and maybe to raise ad spend.
New products show where a competitor is investing before they announce it.
Removed products show what didn't sell.

None of these need a perfect real-time feed. A daily diff catches nearly all of them, and every few hours is plenty in fast categories.

What a competitor's catalog tells you

WooCommerce is one of the most widely used e-commerce platforms, so many of your competitors probably run on it. Their public catalog is far more structured than it looks from the storefront: every product has a current price, a regular price, a sale price, a stock status and usually a SKU, and all of it is visible to any visitor.

Turning that into a reliable feed is the hard part. Themes differ, prices are stored in different currencies and units, variations hide inside parent products, and stores change over time. Our actors handle all of that and are maintained against WooCommerce changes, so you get clean rows instead of a parsing project.

Even a single snapshot is informative. Here are three real public WooCommerce stores on Oct 20, 2026: a 39-product drinkware shop, a 69-product online butcher and a 101-product outdoor-bottle brand.

On the day we checked, 13–36% of products were out of stock and 1–15% were on sale. Those are exactly the fields that change from one day to the next, and the changes are what you want alerts for.

What that's worth to different teams:

E-commerce managers: match or hold prices on overlapping products, and push ads when a rival sells out.
Category and merchandising teams: see new launches and discontinued lines as they happen, not at the next quarterly review.
Agencies and analysts: run the same watch across dozens of stores for clients, with one schedule and one dataset.
Brands with resellers: spot resellers undercutting your price policy.

Step 1: snapshot and diff on a schedule

Reading a catalog once is easy. The useful part is remembering last run's catalog and reporting only what changed. WooCommerce Product Feed & Catalog Monitor does exactly that:

The first run for a store records a baseline and is never charged for "changes".
Every later run compares against the stored snapshot and emits one dataset row per change, typed new, removed, price_changed or stock_changed.

{
  "storeUrls": ["https://competitor-one.com", "https://competitor-two.com"],
  "maxProductsPerStore": 1000,
  "outputMode": "changes",
  "stateName": "competitor-watch",
  "maxChargeUsd": 2
}

A change row looks like this:

{
  "recordType": "change",
  "store": "https://competitor-one.com",
  "changeType": "price_changed",
  "name": "Ceramic Mug - 12oz",
  "sku": "MUG-12",
  "price": 1650,
  "oldPrice": 1400,
  "currency": "USD",
  "stock_status": "instock",
  "permalink": "https://competitor-one.com/product/ceramic-mug-12oz",
  "detectedAt": "2026-10-16T09:00:00+00:00"
}

Keep stateName identical across runs. It names the key-value store that holds the snapshot. Change it only when you want a fresh baseline, for example when you start watching a different set of stores.

Schedule it. In Apify Console, go to Schedules → Create new. Pick the actor, or better, save the input above as a Task first, then set a cron expression:

0 7 * * * runs daily at 07:00, which is enough for most categories;
0 */4 * * * runs every four hours, for fast-moving categories or during sale seasons.

What it costs

Pricing is pay-per-event:

Event	Price
Run started	$0.001
Product checked	$0.0002 ($0.20 per 1,000)
Change detected	$0.005

A 1,000-product competitor checked daily with about 10 changes a day costs roughly $0.25 a run, or about $7.50 a month. Three competitors of the three sizes we probed (39, 69 and 101 products) would cost about $0.04 a run combined before changes. To spend less, lower maxProductsPerStore or the schedule frequency. maxChargeUsd is a hard per-run cap, and stores that fail (no public API, network errors) return an error row that isn't charged.

Step 2: alerts with Zapier, Make or Apify integrations

Option A: Zapier

Trigger: Apify → Finished Task Run. Connect your Apify account and pick the task from step 2.
Action: Apify → Fetch Dataset Items, using the defaultDatasetId from the trigger.
Filter: only continue if recordType exactly matches change and changeType exactly matches price_changed. Add another path for stock_changed or new if you want those too.
Formatter → Numbers → Perform Math Operation: divide price and oldPrice by 100 to show dollars instead of cents.
Action: Slack → Send Channel Message with the formatted values and the permalink.

If a run returns many changes, Zapier sends one message per row by default. Use Formatter → Utilities → Line-item to Text to send a single digest message per run.

Option B: Make

Apify → Watch Task Runs (an instant trigger; Make creates the webhook for you).
Apify → Get Dataset Items, with Dataset ID = defaultDatasetId from module 1.
Add a filter between modules 2 and 3: recordType = change and changeType = price_changed.
Google Sheets → Add a Row (a price-history log), and/or Slack → Create a Message.

Add a Router with one route per changeType. Send new products to a "competitor launches" channel and stock_changed to an out-of-stock channel.

Option C: no automation tool at all

Every Apify task has built-in integrations: Slack, email, Google Drive and webhooks. Open the task, go to Integrations, and add Slack or Send email triggered on Run succeeded. You won't get per-row filtering, but you get a free "the run finished, here's the dataset" notification.

Step 3: the code version

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("dheerajitbhu/woocommerce-catalog-monitor").call(run_input={
    "storeUrls": ["https://competitor-one.com"],
    "outputMode": "changes",
    "stateName": "competitor-watch",
})

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row.get("recordType") == "change" and row["changeType"] == "price_changed":
        old, new = row["oldPrice"] / 100, row["price"] / 100
        pct = (new - old) / old * 100 if old else 0
        print(f"{row['name']}: {old:.2f} -> {new:.2f} ({pct:+.1f}%) {row['permalink']}")

Scheduling, cloud state, many stores per run, error rows instead of crashes, per-run spend caps and the Zapier and Make integrations all come built in.

Which alerts are actually worth sending

An alert for every change becomes noise within a week. Pricing-strategy guides such as Valyupeak's WooCommerce price-tracking setup and Pricerr's monitoring guide make the same point: design alerts around decisions you'd actually act on.

Price drops beyond a threshold on products you also sell. Valyupeak suggests something like 8–15% as the point where a deliberate response is warranted.
Stock-outs on overlapping products. Hold your price and consider raising ad spend.
New products from your main competitors, in a separate, lower-priority channel.
Weekly digest for everything else. Put all changes in a Google Sheet and review it on Mondays.

Set a price floor before you let any of this drive automatic repricing. A competitor's clearance sale shouldn't set your price.

Caveats

Public prices only. Member-only pricing, coupons applied at checkout and B2B price tables aren't in public catalog data.
Currencies and minor units. Prices come back in the store's own currency and minor units. Convert before comparing across stores.
Variable products. For products with variations (sizes, colours), the top-level price may not reflect every variation's price. Check the variations you care about if sizes are priced differently.
Matching your catalog to theirs (which of their products equals which of yours) is a separate problem. SKUs rarely match across brands, so most teams keep a small mapping sheet for their key products.
Be a good citizen. The actor reads catalogs lightly and politely. Daily or every few hours is plenty. Use the data in line with each store's terms.

FAQ

Is it legal to monitor a competitor's catalog?
The actor reads only the public catalog data the store shows any visitor, without login. You're still responsible for following each site's terms and local law, so treat this as public price intelligence, not a way into anything private.

How do I know if a store is on WooCommerce?
Just add its URL. Stores that can't be read come back as an error row and aren't charged.

Does it work for Shopify stores?
No, this actor is WooCommerce-only. Shopify has its own public storefront JSON, which our DriftGuard actor reads for a different job: checking your own Google Merchant Center feed.

How often should I run it?
Daily for most categories. Every 4 hours around big sale events or in fast-moving categories. Each run is charged per product checked, so frequency is the main cost lever.

What happens on the very first run?
It records the baseline snapshot and isn't charged for changes. Alerts start from the second run.

Can I watch several competitors at once?
Yes. Put them all in storeUrls. One run checks every store and diffs each against its own previous snapshot.

Can I monitor my own store with it?
Yes. Some merchants use it as a cheap audit log of their own price and stock changes.

Try it

WooCommerce Product Feed & Catalog Monitor on Apify: add your competitors' store URLs, keep stateName fixed, schedule it daily, and connect Slack. Questions? Email apify@integrationyard.com.

Data in this post: read-only checks of three public WooCommerce stores on Oct 20, 2026, plus the actor's published pricing.

How to Scrape Weibo and Bilibili Without Login in 2026

Integration Yard — Tue, 20 Oct 2026 14:17:41 +0000

TL;DR

You can collect a lot of public Weibo and Bilibili data in 2026 without logging in: search results, comments, hot search, trending videos, creator profiles and danmaku summaries. Between them, our two actors cover 11 no-login modes.

The hard limits are real. Without a session, Weibo search returns only the first page (about 10 posts per tab, about 30 unique posts per keyword per run). Bilibili hides subtitle text and creator upload lists behind login. Work with these limits, not against them: schedule frequent runs, and search instead of browsing a creator's space page.

In a live run on Oct 20, 2026, the Weibo hot search board came back as 50 rows in 9.3 seconds. The #1 topic had 13x the heat of #50.

Engagement is extremely concentrated. In our export of 200 Bilibili comments, the top 10% held 90% of all likes, so sort before you read.

Price on Apify Store: $3 per 1,000 Weibo rows and $2 per 1,000 Bilibili rows with our actors. The most-used alternatives charge $35 and $20 per 1,000.

Weibo (微博) and Bilibili (哔哩哔哩, "B站") are two of the most useful public data sources on the Chinese internet. Weibo is where news breaks and brands get discussed. Its hot search board (热搜) is effectively China's live trending list. Bilibili is the largest long-form video community, with a young, very engaged audience and an overlay comment layer called danmaku (弹幕) that no Western platform has.

If you do market research, brand monitoring, academic work or trend tracking on China, you need both. Most guides start with "log in with Playwright and export your cookies". That works until the account gets rate-limited, the cookie expires in a few days, or someone asks whether you should be scraping under a personal account at all.

This guide takes the other route. It covers what you can reliably get without any login, what you can't, and how to build a monitoring loop around those limits. Every number in it comes from real runs of our actors, our benchmark exports, or the public Apify Store API.

Disclosure: we build and maintain the Weibo Scraper and Bilibili Scraper used here.

Why scrape Weibo and Bilibili
Login vs no login: the honest trade-off
Scraping Bilibili without login
Scraping Weibo without login
What the data looks like
What it costs
Building a monitoring loop
Code: Python and the Apify API
Limits, ethics and data protection
FAQ
Try it

Why scrape Weibo and Bilibili

China had 1.125 billion internet users at the end of 2025, and 602 million of them had used generative AI (CNNIC 57th Statistical Report). Almost none of that conversation happens on platforms Western tools cover. Weibo and Bilibili are two of the biggest public windows into it:

	Weibo (微博)	Bilibili (哔哩哔哩)
Monthly active users	561 million (June 2026)	371 million (Q2 2026)
Daily active users	254 million	116.5 million
Audience	broad, national; news, celebrities, brands, public debate	young: average user age 26.5; 113 minutes a day on the platform
What it's for	breaking news, the hot search board, brand campaigns, public reaction	long-form video reviews, tutorials, gaming, tech, anime; comments and danmaku
Why brands care	real-time share of voice, crisis detection, campaign feedback	Gen Z product opinions, creator (UP主) marketing; top ad verticals are games, consumer electronics, internet services, e-commerce and automotive

Sources: Weibo Q2 2026 results; Bilibili Q2 2026 results and earnings call.

Who uses this data, and for what:

Brands and agencies: track mentions, measure campaign reaction, find the creators who move your category, and catch a PR problem while it's still a comment thread.
Market researchers: compare share of voice, follow topic momentum hour by hour, and benchmark engagement across categories.
AI and LLM teams: fresh, timestamped, opinionated Chinese text for sentiment models, trend summarisation, evaluation sets and RAG over current events.

A few common, concrete use cases:

Brand and product monitoring. What are people saying about 小米, 比亚迪 or your own brand this week? What's in the comments under the top reviews of your product category?
Trend detection. Which topics are rising on the hot search board right now, and which are labelled 新 (new) or 热 (hot)?
Creator research. Which Bilibili creators (UP主) in a niche have the most followers and the most engaged audiences?
Content and sentiment analysis. Comments and danmaku are short, opinionated and timestamped, which makes them good raw material for sentiment models and LLM summaries.
Academic research on public discourse, media and online communities.

All of these need public data at a steady cadence. None of them needs a logged-in account.

Login vs no login: the honest trade-off

Popular Weibo guides, including WebscrapingHQ's 2026 Weibo scraper guide and open-source projects like MediaCrawler, log in with a real account and reuse its cookies. That gets you deep search pagination and full timelines. It also gets you:

cookies that expire within days, which means session-refresh code and monitoring;
account-level rate limits and bans that land on a real person's account;
a weaker legal and ethical position, because you're accessing data as a logged-in user rather than as any visitor.

The no-login approach gives up depth in exchange for reliability and a much simpler compliance story. Here's what it can and can't reach:

	Without login (our actors)	Needs login
Weibo hot search board	✅ full board
Weibo keyword search	✅ first page per tab (~30 unique posts per keyword per run)	deep pagination
Weibo post detail and comments	✅ paged with Weibo's own cursor
Weibo user profile and latest posts	✅ profile, about 10 latest posts	full timeline
Bilibili search	✅ up to ~1,000 results per keyword
Bilibili video stats, comments, replies	✅
Bilibili popular board, creator profiles	✅
Bilibili danmaku	✅ as a per-video summary
Bilibili subtitles / AI subtitles	only `hasSubtitles` true/false	✅ subtitle text
Bilibili creator upload list (space page)	❌ blocked by risk control. Search by creator name instead.	✅

For most monitoring work, the left column is enough. Where it isn't, the fix is usually frequency, not depth: if you can only see the newest 30 posts for a keyword, look every hour.

Scraping Bilibili without login

Bilibili Scraper has five modes:

Mode	What you get	Input
`search`	Videos matching keywords, with sort, length and date filters	`searchQueries`
`video_detail`	Views, likes, coins, favorites, shares, danmaku count, comment count, parts, `hasSubtitles`	`videoUrls` (links or BV ids)
`video_comments`	Top-level comments (popular or newest first), plus optional replies	`videoUrls`
`popular`	The popular / trending board	none
`user_profile`	Creator followers, following, video count, total likes, level, verification, bio	`userIds`

The includeComments, includeReplies, includeDanmaku and includeVideoDetails add-ons work on search, video_detail and popular, so one run can return videos and their comments.

{
  "mode": "search",
  "searchQueries": ["机械键盘"],
  "maxResults": 50,
  "sortOrder": "pubdate",
  "includeComments": true,
  "maxComments": 20
}

Danmaku are the bullet comments that fly across a Bilibili video. A popular video can have thousands. Rather than bill you for each one, includeDanmaku adds one summary row per video: number of danmaku fetched, unique senders, per-minute rate, peak minute and a timeline sample. That's usually what you want for analysis anyway, because the peak minute tells you which moment of the video people reacted to.

Plan around three Bilibili limits:

Subtitles need login. The actor reports hasSubtitles but can't download the text.
Creator upload lists need login. A creator's space page is blocked by risk control for anonymous visitors. To find a creator's videos, search for their name.
Search tops out at about 1,000 results per keyword. Split broad topics into narrower keywords or date windows.

Here's what the popular board looked like on the morning of Oct 20, 2026, from one popular run of 50 videos:

The median video had about 535,000 views and was about 6.5 minutes long. Viewers left a median 76 likes per 1,000 views, but only about 3 comments and 2 danmaku. Likes measure reach. Comments and danmaku are rarer and say much more, which is why most research projects want them.

Scraping Weibo without login

Weibo Scraper has six modes:

Mode	What you get	Input
`search`	Posts from the 综合 / 实时 / 热门 tabs, merged and deduplicated	`searchQueries`
`hot_search`	The live hot search board: rank, keyword, heat, label	none
`post_detail`	Full text, counts, images, hashtags and author	`postUrls`
`post_comments`	Comments, paged with Weibo's own cursor	`postUrls`
`user_profile`	Followers, following, post count, verification, bio	`userIds`
`user_posts`	A user's latest posts	`userIds`

The main limit: without a logged-in account, Weibo serves only the first page of search results. That's about 10 posts per tab, so up to about 30 unique posts per keyword per run after merging the three tabs. User timelines stop at about 10 latest posts. Comments, post details, profiles and the hot search board aren't limited this way.

{ "mode": "search", "searchQueries": ["小米", "比亚迪"], "includeComments": true, "maxComments": 20 }

{ "mode": "hot_search", "maxResults": 50 }

The hot search board is the fastest, cheapest signal Weibo offers. Our live run at 7:02 AM PT on Oct 20, 2026 returned 50 rows in 9.3 seconds, with no blocks:

Two things stand out. First, the curve is steep: #1 had 1,875,414 heat and #50 had 145,560, almost 13x less, and the top 10 entries held 41% of the board's total heat. Second, half the board (25 of 50) was labelled 新 (new). The board turns over fast, so snapshot it hourly if you want to catch topics on the way up rather than at the peak.

What the data looks like

Each actor writes one row per item to an Apify dataset. Rows have a type field, so posts, comments and replies can share a dataset:

{ "type": "comment", "postId": "5355477305394531", "commentId": "5355484431519240",
  "text": "…", "likesCount": 3, "replyCount": 0, "authorName": "…", "authorId": "…",
  "createdAt": "2026-10-18T…", "scrapedAt": "2026-10-18T…" }

Some shape numbers from our 882-row benchmark export (Oct 18, 2026):

Row type	Rows	Fields per row	Median text length
Weibo post	20	22	70.5 characters
Weibo comment	99	10	18 characters
Bilibili comment	200	11	25 characters
Bilibili reply	206	11	16.5 characters

Chinese comments are short. A median of 18–25 characters is a phrase, not a paragraph. That matters for analysis: one comment rarely carries enough context alone, so models do better with many comments together.

Likes are also very unevenly spread:

In our Bilibili export, the top 10% of comments held 90% of all likes, and the median comment had 2 likes. Weibo comments were less extreme, but the top 10% still held 66%. In practice:

Sort by likes first if you're reading by hand or sending a sample to an LLM. The top 20 comments are what most viewers actually saw.
Don't average likes. The mean is dominated by a few outliers. Use medians or percentiles.
Use popular ordering for Bilibili comments when you want the consensus view, and newest-first when you're monitoring for new complaints.

If you're sending these rows to an LLM, both actors also support "outputFormat": "compact". In our token benchmark it cut tokens by 41.9% for Bilibili and 24.8% for Weibo versus pretty JSON.

What it costs

Both actors use pay-per-event pricing. You pay per row returned, and error rows are free:

	Weibo Scraper	Bilibili Scraper
Run start	$0.001	$0.001
Row (post, comment, reply, profile, hot-search entry, video)	$0.003 ($3 per 1,000)	$0.002 ($2 per 1,000)
Danmaku summary	n/a	$0.01 per video
Per-run cap	`maxChargeUsd` (default $5)	`maxChargeUsd` (default $5)

Example costs: 100 Bilibili search results are $0.20. Ten videos with 50 comments each come to 510 rows, about $1.02. Hourly Weibo hot search (50 rows) costs about $0.15 a run.

For context, here's what the most-used alternatives on Apify Store charge per result as of today:

These are per-result event prices from the public Apify Store API on Oct 20, 2026. Run-start fees and proxy costs aren't included, and the actors don't return identical fields, so compare what each one outputs as well as the price. The most-used Weibo actor charges $35 per 1,000 and the most-used Bilibili actor $20 per 1,000.

Building a monitoring loop

Because no-login Weibo search only shows the newest page, the winning pattern is small, frequent runs plus deduplication:

Save each input as an Apify Task: "brand-weibo-search", "hot-search", "category-bilibili-search".
Schedule them. Use 0 * * * * (hourly) for Weibo search and hot search, and 0 6 * * * (daily) for Bilibili search and comments.
Deduplicate on id. Use id for Weibo posts, commentId for comments and bvid for Bilibili videos. An hourly run will return many posts you already have, and that's expected.
Send the results somewhere. Use Apify's Google Sheets or webhook integrations, or Zapier or Make, or pull the dataset from your own code.
Summarize. A daily job reads the last 24 hours of rows and writes a trend or sentiment report. This is a good use of an LLM.

At 24 hourly hot-search runs a day, that's about $3.60 a day for a complete hourly record of China's trending list.

Code: Python and the Apify API

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

# Bilibili: search + top comments
bili = client.actor("dheerajitbhu/bilibili-scraper").call(run_input={
    "mode": "search", "searchQueries": ["原神"], "maxResults": 30,
    "includeComments": True, "maxComments": 20,
})
rows = list(client.dataset(bili["defaultDatasetId"]).iterate_items())
comments = sorted((r for r in rows if r.get("type") == "comment"),
                  key=lambda r: r.get("likeCount") or 0, reverse=True)
for c in comments[:10]:
    print(c["likeCount"], c["text"][:60])

# Weibo: hot search board
weibo = client.actor("dheerajitbhu/weibo-scraper").call(run_input={
    "mode": "hot_search", "maxResults": 50,
})
for row in client.dataset(weibo["defaultDatasetId"]).iterate_items():
    print(row.get("rank"), row.get("keyword"), row.get("heat"), row.get("label"))

Both actors run a daily self-test against the live sites. If Bilibili or Weibo blocks every request in a run, the run is marked failed instead of quietly finishing "green" with zero rows, so your schedule's failure alert actually fires. Both are maintained daily against the platforms' site changes and handle their access protections for you.

Limits, ethics and data protection

Public data only. Neither actor ever logs in. Comment and post rows carry only the author's public nickname and numeric id. No avatars, no IP-region tags, no private fields.
Personal data is still personal data. Nicknames and ids can identify people. If you're subject to GDPR, PIPL (China's Personal Information Protection Law) or similar rules, aggregate where you can, and keep only what your purpose needs.
Be gentle. Hourly is plenty for monitoring. The actors pace their requests. Don't run 50 copies in parallel.
Respect the platforms' terms and your own legal advice for your use case. This post explains what's technically reachable without login. It isn't legal advice.

FAQ

Can I scrape Weibo without an account in 2026?
Yes, for the hot search board, comments, post details, profiles and the first page of search results. Deep search pagination and full timelines need a logged-in session, which these actors deliberately don't use.

Why do I only get about 30 posts per Weibo keyword?
That's what Weibo serves to anonymous visitors: one page from each of the 综合, 实时 and 热门 tabs, merged and deduplicated. Schedule hourly runs and deduplicate on post id to build a continuous feed.

Can I download Bilibili subtitles or danmaku text?
Subtitle text needs login, so the actor reports hasSubtitles instead. Danmaku come as a per-video summary (count, unique senders, per-minute rate, peak minute, timeline sample) billed at $0.01 per video.

How do I get all videos from one Bilibili creator?
The creator's space page is blocked for anonymous visitors. Use search with the creator's name, and use user_profile for their follower and video counts.

Do I need Chinese proxies?
No. Access is handled for you inside the actors, with nothing to configure. Our live hot search run finished in 9.3 seconds with no blocks.

How fresh is the data?
It's fetched live at run time. scrapedAt is on every row.

Can I feed the output to an LLM?
Yes. Set "outputFormat": "compact" to cut input tokens (by 42% for Bilibili and 25% for Weibo in our benchmark), and sort by likes before sampling.

Try it

Weibo Scraper: search, hot search, comments, profiles. $3 per 1,000 rows.
Bilibili Scraper: search, video stats, comments, popular, creators, danmaku summaries. $2 per 1,000 rows.

Leave the inputs at their defaults for a first run, or paste one of the JSON examples above. Questions or a mode you need? Email apify@integrationyard.com.

Data in this post: live runs on Oct 20, 2026 (Weibo hot search run alIwu4CFqqhdwEd2T, Bilibili popular, 50 videos), our 882-row benchmark export from Oct 18, 2026, and the public Apify Store API on Oct 20, 2026.

Compact, Lossless JSON Output for LLM Agents: How Much Do You Actually Save?

Integration Yard — Tue, 20 Oct 2026 07:53:13 +0000

TL;DR

We measured 882 real rows scraped by five of our Apify actors in three encodings: pretty JSON, minified JSON and our lossless compact output.

With OpenAI's o200k_base tokenizer (GPT-4o / GPT-5 family), compact output used 33.8% fewer tokens than pretty JSON and 18.3% fewer than minified JSON.

Savings per actor ranged from 43% (AliExpress) down to 13% (WeChat Articles). Short, repetitive rows win big; long free-text rows barely move.

Every file passed a lossless round-trip check. No rows or fields dropped.

At 300,000 rows a month on GPT-5.4, that is about $171 vs $113 in input tokens, or roughly $58 a month saved for changing one input flag.

If you've ever pasted a scraper export into ChatGPT or sent one to an agent as a tool result, you've paid for the same key names hundreds of times. "title", "likeCount", "authorName", "scrapedAt": in an array of JSON objects every row repeats every key, and the model bills you for each copy before it reads a single value.

That cost is easy to ignore on one prompt and hard to ignore in production. An agent that pulls a few thousand rows a day, re-reads them on every turn, and runs on a frontier model can spend a real share of its budget on punctuation and repeated field names.

This post is the full benchmark behind the outputFormat: "compact" option in our Bilibili, Weibo, WeChat Articles, AliExpress and Douban actors on Apify. It covers what the format is, how we measured it, the per-actor results (including the ones that don't look good), what that means in dollars at current prices, when you should not use it, and how to plug it into LangChain or a plain LLM call.

Disclosure: we build and maintain these actors.

Why tokens cost money
The format: what changes, and what doesn't
Methodology
Results by actor
What it saves in dollars
How this compares to TOON, CSV and lossy compressors
When not to use it
Using it with LangChain and LLM pipelines
FAQ
Try it

Why tokens cost money

LLM APIs bill per token, and input tokens are usually most of the bill for data-heavy work. Your prompt holds the data, and the answer is short. Here are list prices per million input tokens as of October 2026:

Model	Input $ / 1M tokens	Output $ / 1M tokens	Source
OpenAI GPT-5.4	$2.50	$15.00	OpenAI pricing
OpenAI GPT-5	$1.25	$10.00	OpenAI pricing
OpenAI GPT-5 mini	$0.25	$2.00	OpenAI pricing
Anthropic Claude Sonnet 5.5	$2.00	$10.00	Anthropic pricing

A token is not a character or a word. Tokenizers split text into chunks they've learned are common. That means JSON's structure isn't free: quotes, colons, commas, braces, the newlines and indentation in pretty-printed JSON, and above all the repeated key names all become tokens.

Here's a quick way to see it. In our benchmark, one pretty-printed row averaged 228 tokens with the o200k_base tokenizer. The same row averaged 185 tokens minified and 151 tokens in compact form. The data didn't change. Only the packaging did.

Three things make this worse in agent workloads than in one-off prompts:

Re-reading. An agent loop sends the whole conversation, tool results included, back to the model on each turn. A 40,000-token tool result read five times costs 200,000 input tokens.
Context limits. Even with very large context windows, fewer tokens per row means more rows fit before you hit the limit or have to chunk.
Latency. Prompt processing time grows with input length, so a smaller prompt also comes back faster.

Prompt caching and batch discounts help too, and they stack with what follows. Fewer tokens is still fewer tokens, whether they're billed at the full or the cached rate.

The format: what changes, and what doesn't

A normal scraper export is an array of JSON objects, and every row repeats every key name:

[
  {"id": "BV1xx", "title": "2026 keyboard roundup", "views": 12034},
  {"id": "BV2yy", "views": 877}
]

Compact output is our token-optimized encoding of exactly the same rows. It removes the structural repetition that LLMs bill you for, and keeps everything that matters:

Lossless. Every value, type and null survives, and so does the difference between a field that's null and a field that's absent. Plenty of "compact" formats quietly lose that last one.
Still JSON. Any JSON parser reads it. No new parser, no new library.
Robust on messy, real data. Mixed row shapes, Chinese text, nested lists and unusual characters are all handled. We verify the round trip on every format change.
Additive. Your normal dataset is untouched. The compact copy is an extra record on the same run, at no extra cost.

Getting all of that right at once is harder than it looks, which is why we built it into the actors rather than leaving it as an exercise.

Methodology

We wanted numbers we'd be comfortable defending, so the benchmark is deliberately boring:

Real data, not synthetic. 882 rows exported from real runs of five actors: AliExpress (130 rows), Bilibili (406), Douban (150), Weibo (119) and WeChat Articles (77). They're mostly flat rows (a few fields hold small lists), and some exports mix row shapes, such as Bilibili comments next to replies and Weibo posts next to comments, because that's what real exports look like.
Three encodings of the exact same rows. Pretty JSON (json.dumps(rows, indent=2, ensure_ascii=False)), minified JSON (separators=(",", ":")), and compact output, exactly as the actor writes it to the OUTPUT_COMPACT key.
A lossless check first. Each file had to round-trip back to the exact original rows before we measured it. A format that drops data can always look smaller, so this check comes first.
Two real tokenizers. tiktoken with o200k_base (GPT-4o, GPT-4.1 and GPT-5 family) and cl100k_base (GPT-4 / GPT-3.5). The headline numbers use o200k_base. cl100k_base came out within about two points (31.7% vs pretty, 17.1% vs minified).
Two baselines. Many "X% smaller than JSON" claims compare against pretty JSON, which is the easy baseline. We report both, and we think minified JSON is the honest one, because it's what a careful pipeline already sends.

What we did not measure: downstream answer accuracy, and non-OpenAI tokenizers such as Claude's, Llama's or Qwen's. Independent TOON benchmarks show savings vary by tokenizer, so measure on the model you actually use.

Results by actor

Actor	Rows	Pretty	Minified	Compact	vs pretty	vs minified
AliExpress Scraper	130	42,425	32,461	24,154	43.1%	25.6%
Bilibili Scraper	406	67,672	53,034	39,320	41.9%	25.9%
Douban Scraper	150	26,632	20,696	16,091	39.6%	22.3%
Weibo Scraper	119	22,611	17,879	17,010	24.8%	4.9%
WeChat Articles Scraper	77	42,044	39,061	36,645	12.8%	6.2%
All five	882	201,384	163,131	133,220	33.8%	18.3%

Token counts with o200k_base.

Three patterns explain the spread:

1. Short values plus many fields means big wins (AliExpress, Bilibili, Douban: 40–43%). Product listings, video comments and book or film entries have lots of short fields: IDs, prices, counts, timestamps, short titles. Structure is a large share of each row, so removing the repetition removes a large share of the tokens. Bilibili gets an extra boost from row count: the more rows, the bigger the saving.

2. Mixed row shapes cut into the gain (Weibo: 25% vs pretty, under 5% vs minified). A Weibo export mixes 22-key posts with 10-key comments. Rows of very different shapes share less structure, so there's less repetition to remove. If you filter to one row type, you'd recover most of the gap.

3. Long free text leaves little to remove (WeChat Articles: 13%). WeChat rows are mostly article titles and summaries in Chinese, so the values already dominate the token count. Removing repetition still helps, but there's not much structure left to strip.

Bytes tell the same story, with slightly larger gains: 40–46% fewer UTF-8 bytes for AliExpress, Bilibili and Douban against pretty JSON. That matters if you store or move exports at volume.

What it saves in dollars

Here's a worked example using the measured per-row averages: 228.3 tokens pretty, 185.0 minified and 151.0 compact.

Say an agent ingests 10,000 scraped rows a day (about 300,000 a month) and reads each row once:

	Pretty JSON	Minified JSON	Compact
Input tokens / month	68.5M	55.5M	45.3M
GPT-5.4 at $2.50 / 1M	$171.24	$138.72	$113.28
GPT-5 at $1.25 / 1M	$85.62	$69.36	$56.64
GPT-5 mini at $0.25 / 1M	$17.12	$13.87	$11.33

On GPT-5.4, switching from pretty JSON saves about $58 a month (~$696 a year). Switching from minified saves about $25 a month. If the agent re-reads the data across several turns, multiply both by the number of reads. On a mini model the dollar savings are small, but you can fit about a third more rows into the same context.

These are input costs only, at OpenAI list prices from October 2026, and they use the OpenAI tokenizer we measured. Claude uses a different tokenizer, so check your own counts before applying these percentages to Claude prices.

How this compares to TOON, CSV and lossy compressors

Token-efficient formats are a crowded topic right now, so it's worth placing this one.

TOON (Token-Oriented Object Notation) is the best-known "JSON for LLMs" format. It uses a YAML-like layout with tabular arrays. Independent benchmarks report about 34% fewer tokens than minified JSON on uniform tabular data, but 14–17% *more* on nested or semi-uniform data, and smaller savings on non-OpenAI tokenizers. The author also notes that the widely quoted "40–60%" figure is measured against pretty JSON. Our numbers sit in the same range for the same reason. Tabular data compresses well, and mixed shapes don't.
CSV is very compact for flat tables, but it has no types (is "123" a string or a number?), no null, and no way to say "this key was absent". The TOON project's own benchmarks exclude CSV from parts of their accuracy tests because it can't represent the non-flat datasets.
Lossy "AI compressors" can post bigger headline numbers by summarizing or sampling. We tested one (headroom) on the same kind of files, and it dropped rows or fields on 7 of 8 sample datasets. That's fine for an agent's scratch memory, but not for data a customer is paying to receive intact.
Agent-loop caveat. A 2026 study of token-optimized formats in agentic systems found that formats which shrink tool outputs the model must write can increase total cost when parsing failures trigger extra turns. Compact output is an input format: the model reads it and never has to produce it.

We built compact output to be plain JSON that keeps types and null and round-trips exactly, so it's safe for data a customer is paying for.

When not to use it

Compact output isn't always the right call. Skip it when:

You're loading a spreadsheet, BI tool or database. Use the normal dataset export in CSV, Excel or JSON. The compact format exists to cut LLM tokens, not to replace ordinary exports.
Your rows are mostly long text. As the WeChat result shows, savings drop toward 10%, and per-row objects can be easier to debug.
You're mixing very different row types in one table. Weibo's posts plus comments showed this. Either filter to one row type, or accept the smaller gain.
You need row-level retrieval. For RAG, you usually embed one row or chunk at a time. Each chunk is small, so the saving shrinks. Compact output pays off when you send many rows together in one prompt.
The model has to produce this format. Keep model outputs in ordinary JSON or structured outputs. Use compact for what the model reads.
You already minify, and your volume is tiny. At a few hundred rows a month, an 18% saving on minified JSON is worth less than your time.

Using it with LangChain and LLM pipelines

Set "outputFormat": "compact" on the run. The dataset is unchanged and so is the price. The actor also writes the compact copy to the run's key-value store under OUTPUT_COMPACT.

1. Fetch it with the Apify client

import json
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("dheerajitbhu/aliexpress-scraper").call(run_input={
    "mode": "search",
    "searchQueries": ["mechanical keyboard"],
    "maxResults": 100,
    "outputFormat": "compact",
})
record = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("OUTPUT_COMPACT")
payload = record["value"]  # compact copy of the run's rows

(Field names in run_input vary by actor. Check each actor's input tab.)

2. Send it straight to the model

from openai import OpenAI

prompt = (
    "Below are AliExpress product rows in compact JSON.\n\n"
    + json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
    + "\n\nWhich five products have the best rating-to-price ratio?"
)
resp = OpenAI().chat.completions.create(
    model="gpt-5", messages=[{"role": "user", "content": prompt}])
print(resp.choices[0].message.content)

3. Use it as an agent tool result in LangChain

from langchain_core.tools import tool

@tool
def latest_products(query: str) -> str:
    """Return current AliExpress listings for a query in compact JSON."""
    run = client.actor("dheerajitbhu/aliexpress-scraper").call(run_input={
        "mode": "search", "searchQueries": [query], "maxResults": 50, "outputFormat": "compact"})
    rec = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("OUTPUT_COMPACT")
    return json.dumps(rec["value"], ensure_ascii=False, separators=(",", ":"))

Because tool results are re-sent on every later turn of the agent loop, the savings repeat every time the agent reads that result again.

4. Need regular dicts? For pandas, LangChain Documents or a vector store, read the normal dataset from the same run. It's always there, unchanged:

from langchain_core.documents import Document
rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())
docs = [Document(page_content=r.get("title", ""), metadata=r) for r in rows]

FAQ

Is it really lossless?
Yes, within what JSON can represent. Every benchmark file round-tripped back to the exact original rows, including real nulls and keys absent from some rows.

Does compact output change what I pay for the actor?
No. The run pushes the same rows to the dataset at the same per-result price. OUTPUT_COMPACT is an extra key-value store record.

Do LLMs understand the compact shape?
In our use, yes, It reads like a table. We haven't run a formal accuracy benchmark, so test your own task if accuracy is critical.

Why not just use CSV?
CSV loses types and can't tell null from an absent key. Compact output keeps both and still parses with json.loads.

Does this work with Claude, Gemini or open models?
The format works with any model. The percentages above come from OpenAI tokenizers, and other tokenizers will give different numbers. The structural saving, writing keys once instead of N times, applies everywhere.

Why are your numbers smaller than some format headlines?
Big headline numbers usually come from comparing against pretty-printed JSON on perfectly uniform synthetic data. On 882 real, messy rows we measured 13–43% per actor against pretty JSON and 18% overall against minified. We'd rather quote the smaller, real number.

Try it

Compact output is available on five Integration Yard actors on Apify. Add "outputFormat": "compact" to any run:

AliExpress Scraper: 43% fewer tokens than pretty JSON in our benchmark
Bilibili Scraper: 42%
Douban Scraper: 40%
Weibo Scraper: 25%
WeChat Articles Scraper: 13%

Apify's free plan includes monthly platform credit, so you can run it on your own query and count the tokens yourself. If your numbers differ from ours, email apify@integrationyard.com. We'll publish corrections.

Benchmark data: 882 rows, tiktoken o200k_base and cl100k_base, October 2026. Prices are list prices checked October 20, 2026.

DEV Community: Integration Yard

Catch Google Merchant Center Disapprovals Before Google Does

Contents

What Google actually checks

Why feeds drift from the storefront

How exposed is a real catalog? Data from four stores

The manual audit (do this once, today)

Who this protects, and what it's worth

Automate it: storefront vs feed, every day

Setup

Get alerted

What it costs

Caveats

FAQ

Try it

Weibo Hot Search (微博热搜) Data in 2026: Track China's Trending Topics Without Code

Contents

Why 微博热搜 matters

The board in one chart: heat is front-loaded

From a trending topic to the posts behind it

Who uses this data

What you get

Scheduling: building a trend history

FAQ

Try it

Get an Alert When a Competitor Changes Prices on Their WooCommerce Store

Contents

Why competitor price monitoring matters

What a competitor's catalog tells you

Step 1: snapshot and diff on a schedule

What it costs

Step 2: alerts with Zapier, Make or Apify integrations

Option A: Zapier

Option B: Make

Option C: no automation tool at all

Step 3: the code version

Which alerts are actually worth sending

Caveats

FAQ

Try it

How to Scrape Weibo and Bilibili Without Login in 2026

Contents

Why scrape Weibo and Bilibili

Login vs no login: the honest trade-off

Scraping Bilibili without login

Scraping Weibo without login

What the data looks like

What it costs

Building a monitoring loop

Code: Python and the Apify API

Limits, ethics and data protection

FAQ

Try it

Compact, Lossless JSON Output for LLM Agents: How Much Do You Actually Save?

Contents

Why tokens cost money

The format: what changes, and what doesn't

Methodology

Results by actor

What it saves in dollars

How this compares to TOON, CSV and lossy compressors

When not to use it

Using it with LangChain and LLM pipelines

FAQ

Try it