DEV Community: DEUS Automations

Your CDN has an opinion about AI crawlers, and it is not the one you set

DEUS Automations — Sat, 19 Sep 2026 22:31:36 +0000

Everyone reads the headline as "Cloudflare blocks AI crawlers." The measurement says something
narrower and more interesting: on the sites whose own robots.txt says yes, Cloudflare refuses
Amazon's and Meta's crawlers far more often than it refuses OpenAI's or Anthropic's — and for
OAI-SearchBot it refuses slightly less often than sites with no CDN at all.

All figures re-derived from the live Agent Web Index
aggregate on 2026-09-19: 47,315 domains from the Tranco list with a measured verdict, 46,868 of them
with the edge vendor read from the response headers. Dataset CC BY 4.0 (CSV on the hub).

What is being counted

One unit: a domain × crawler pair whose robots.txt allows that crawler. For each pair we made
a real GET with that crawler's published user-agent. If robots.txt says yes and the server says no
— 403, challenge, block page — that refusal contradicts the site's own stated policy, and somebody
other than the site owner made it.

The edge vendor comes from the headers of the same response (cf-ray, akamai-grn, x-amz-cf-id,
x-fastly-request-id…), so it costs zero extra requests. Sites with no recognisable signature form
the control group: "no known edge", 12,384 domains.

Cloudflare, per crawler

26,543 domains behind Cloudflare. 13.1% of the 119,112 robots-allowed pairs are refused anyway. But
that single number hides the whole story:

Crawler	Cloudflare	No known edge	Ratio
Amazonbot	29.9% (2,115/7,062)	13.8% (922/6,677)	2.17×
Meta-ExternalAgent	20.7% (1,471/7,120)	12.4% (832/6,727)	1.67×
Bytespider	26.1% (80/306)	20.1% (37/184)	1.30×
GPTBot	14.0% (3,593/25,736)	11.9% (1,357/11,448)	1.18×
ClaudeBot	14.6% (3,768/25,873)	13.0% (1,511/11,644)	1.12×
Applebot	9.1% (29/320)	7.3% (14/192)	1.24×
PerplexityBot	8.8% (2,314/26,302)	8.2% (973/11,885)	1.07×
OAI-SearchBot	8.5% (2,239/26,393)	9.0% (1,087/12,040)	0.94×

Read the right-hand column, not the left. Sites block AI crawlers for their own reasons everywhere,
CDN or not; the ratio isolates what being behind Cloudflare adds. For ChatGPT's search crawler it
adds nothing. For Anthropic and Perplexity it adds a few percent. For Amazon's crawler it roughly
doubles the chance of being turned away from a page the site said it could have.

The shape tells you whether a human decided

Run the same breakdown on the other vendors and the contrast is the finding:

Edge	Domains	Refusal rate	Widest gap between crawlers	Every crawler, vs control
Akamai	327	42.2%	1.1×	3.2–4.9×
Google (Cloud/GFE)	763	36.6%	1.3×	2.2–4.4×
Sucuri	61	21.4%	21.0×	0.2–3.3×
AWS CloudFront	3,111	16.5%	1.2×	1.3–1.8×
Cloudflare	26,543	13.1%	3.5×	0.9–2.2×
Fastly	1,454	8.3%	1.7×	0.6–0.8×
Vercel	570	5.0%	1.4×	0.4–0.6×
Netlify	197	2.3%	3.2×	0.1–0.3×

Read the last two columns together. Akamai refuses far more traffic than Cloudflare — and refuses
every AI crawler at almost exactly the same rate, roughly three to five times the control on all of
them. That is a wall nobody aimed: a bot-management default that never learned these user-agents
apart, and it is by far the most punishing thing that can sit in front of a site that said yes.
Cloudflare refuses less overall, but selectively, and selectivity is the signature of a list — a
set of names, maintained by someone, applied on behalf of 26,543 sites that mostly never opened the
setting.

Sucuri is the cleanest example of a list there is: ClaudeBot 35%, GPTBot 29%, OAI-SearchBot 30%,
PerplexityBot 2%. Twenty-one to one, across 61 domains whose own robots.txt allows all four.
Nobody writes that shape by accident.

And the bottom of the table is a finding in its own right: behind Fastly, Vercel, Netlify, Imperva,
Azure Front Door or Varnish, a robots-allowed crawler is refused less often than on a site with no
recognisable edge at all — down to 0.1× for Netlify. The edge is not one thing. Two vendors apart on
this table differ more, for a site owner who wanted to be readable, than most of the choices that
owner actually agonises over.

Why this is hard to see

You cannot get these numbers from robots.txt, and you cannot get them from a CDN's own
transparency page, which only covers its own customers and never compares them to a control. You
have to make the request as each crawler, from outside, at scale, and keep the response headers. The
index does exactly that, continuously; every cell above is a live aggregate, not a one-off study.

The per-vendor × per-crawler table is on the hub at
/ai-readiness/#edge, with the pair count
behind every cell. There is also an MCP endpoint, no key, if you would rather have your assistant
pull it: POST https://shop.lumnika.com/ai-readiness/mcp, tool edge_blocking.

Limits, stated first

One observation point (Europe) and the homepage only. A vendor whose rules vary by region or by path will read differently from elsewhere.
Columns do not share a denominator. Amazonbot and Meta-ExternalAgent were added to the probe later, so their columns cover the domains measured since — a more recent slice of the same list, not a different list. Compare within a row first.
Small vendors, small denominators. Sucuri's 21× gap rests on 321 robots-allowed pairs across 61 domains, Akamai's rates on 1,582 pairs across 327. The direction is unambiguous at that size; the second decimal is not. Cloudflare (119,112 pairs) is the only cell where precision is cheap.
"No known edge" is an upper bound, not a vendor. A domain measured before a signature was added to the table stays in that bucket until it is re-requested; a re-request of 250 of them found 14% already carrying a recognised signature. So the named vendors are undercounts.
For part of the domains the edge was read in a later pass than the crawler verdicts, so a site that changed CDN in between appears under its current one.
Google-Extended and Applebot-Extended are robots.txt tokens, not crawlers — they make no HTTP request of their own and are excluded from every number here about server behaviour.

If your site is in the table and you did not choose this, the per-domain page shows exactly which
crawler got which answer: https://shop.lumnika.com/ai-readiness/<your-domain>.

Three PWA decisions that survived a real dining room

DEUS Automations — Sat, 19 Sep 2026 21:22:11 +0000

A point-of-sale web app in a busy restaurant is a hostile environment for the
assumptions most PWA tutorials make. The network drops behind the walk-in
fridge. Four devices edit the same table at the same time. And the one thing a
waiter will never do is wait for your app to finish reloading while a customer
is mid-sentence.

Here are three decisions from Servito, an
order-taking PWA for restaurants and pubs, that only look obvious after you've
watched the app being used during a dinner service.

1. Never call `skipWaiting()` automatically

The default snippet everyone copies makes a new service worker take over as
soon as it installs:

self.addEventListener("install", () => self.skipWaiting());

In a dining room this is a bug with a uniform on. A waiter is halfway through
a five-item order; you ship a deploy; the new worker activates, the page is
reloaded under them, and the order is gone. The customer is still talking.

So the install handler precaches and stops there. The new worker sits in
waiting until a client explicitly asks for it:

self.addEventListener("message", (event) => {
  if (event.data === "SKIP_WAITING") self.skipWaiting();
});

The app surfaces a small "update available" banner, and the update happens when
the staff taps it — between tables, not between courses. Shipping becomes
boring, which is the goal.

2. Scope realtime channels to the venue, not the table

Multi-device is the whole point: the tablet at the pass, the phone in the
waiter's apron, the screen in the kitchen. Supabase Realtime makes the
subscription itself trivial:

const channel = supabase.channel(`floor-plan-${venueId}`);
channel.on(
  "postgres_changes",
  { event: "*", schema: "public", table: "orders", filter: `venue_id=eq.${venueId}` },
  () => refresh()
);

Two details matter more than the API surface.

The filter is server-side. venue_id=eq.${venueId} means the database
never pushes another restaurant's rows onto this socket. Filtering in the
client is not a performance optimisation you can skip — it's the difference
between a leak and a query.

One channel per view, named after what it shows. floor-plan-${venueId},
status-lanes-${venueId}. When something is stale in production, the channel
name tells you which screen to open. A single mega-channel that fans out to
every component is easy to write and impossible to debug at 9pm on a Saturday.

3. Offline means "degrade to something usable", not "queue everything"

Full offline write support for a POS is a distributed-systems project:
conflicting edits to the same bill, out-of-order sync, staff who reconcile
totals by hand anyway. What actually helps is a precached shell and an honest
offline page:

const PRECACHE_URLS = ["/", "/offline", "/manifest.webmanifest"];

The app tells the user the truth immediately instead of accepting input it may
silently lose. "The connection dropped, this order was not saved" is a
recoverable moment. "Your order was saved" followed by a missing order is how
you lose a customer for good — and in a restaurant, a lost order is a lost
table, not a lost row.

The pattern underneath

All three are the same trade: give up an optimisation that reads well in a
blog post, in exchange for behaviour that survives a room full of people who
did not agree to be your test users.

If you want to poke at the result, the live demo
runs without an account — it's the same build, with seeded data.

Most AI-crawler blocking doesn't happen in robots.txt

DEUS Automations — Sat, 19 Sep 2026 20:19:52 +0000

Most AI-crawler blocking doesn't happen in robots.txt

Numbers re-derived from the live Agent Web Index aggregate on 2026-09-19 (47,314 domains with a
measured verdict). Dataset CC BY 4.0: Zenodo DOI, GitHub DeusAcc/agent-web-index, Hugging Face
DeusHorizon/agent-web-index. Live hub: https://shop.lumnika.com/ai-readiness/?src=devto

Everyone audits robots.txt. We probed 47,314 domains the way an AI assistant actually fetches
them — one request from a browser user-agent, then one from each published crawler user-agent —
and compared what robots.txt permits with what the server really returns.

22.5% of measured domains block at least one major AI crawler. Most of those blocks are not in
robots.txt.

Crawler	Served	Blocked in robots.txt	Blocked by the edge anyway	Edge : robots
ClaudeBot	39,946	2,077	6,571	3.2 : 1
GPTBot	40,150	2,595	6,250	2.4 : 1
PerplexityBot	42,518	1,233	4,391	3.6 : 1
OAI-SearchBot	42,507	826	4,513	5.5 : 1

"Blocked by the edge" means one thing only, and it is measured, not inferred: robots.txt allows
the bot, a browser user-agent gets the page, and the crawler user-agent gets a different answer —
a 403, a challenge, or a block page. Nobody wrote that rule for ClaudeBot. It came with a CDN
setting.

It depends on who answers for your domain

Grouping by the edge vendor that serves the host, over crawler×domain pairs:

Edge	Domains	Crawler pairs blocked
Akamai	266	43.0%
Google (Cloud/Frontend)	722	38.1%
Sucuri	52	18.7%
AWS CloudFront	2,903	16.4%
Cloudflare	25,404	12.6%
No identifiable edge	11,335	11.0%
Azure Front Door	283	8.7%

A site behind Akamai is roughly three and a half times more likely to refuse an AI crawler than a
site behind Cloudflare — while both owners believe their robots.txt is the policy.

Mean readability score across measured domains: 73.1 (median 74).

Check yours

Every domain has a page with its own evidence: https://shop.lumnika.com/ai-readiness/<domain>?src=devto.
The same data is a public MCP server, no key, if you would rather have your assistant ask:

POST https://shop.lumnika.com/ai-readiness/mcp
{"jsonrpc":"2.0","id":1,"method":"tools/call",
 "params":{"name":"domain_readiness","arguments":{"host":"example.com"}}}

Limits, stated by us first

Single vantage point, one probe per crawler per domain, one point in time — a challenge page can
be transient and we count it as a block. Percentages cover only domains that answered at all;
unreachable domains are excluded, not counted as open. google-extended and applebot-extended
are robots-only signals with no fetch to compare, so they carry no edge figure. Vendor rows with
few domains (Sucuri, Akamai) have wide error bars — the direction is solid, the second decimal is
not.

Your site's robots.txt says AI crawlers are welcome. Your CDN might disagree.

DEUS Automations — Tue, 15 Sep 2026 23:28:55 +0000

Every week another AI crawler shows up in your logs: ClaudeBot, GPTBot, OAI-SearchBot, PerplexityBot, Google-Extended. Your robots.txt might allow all of them — and your CDN might still be answering with a 403 challenge page before the crawler ever sees your content. Nobody checks for that contradiction, because it means actually making the requests, not just parsing a text file.

So I built AgentReady: you give it a URL, it makes six real HTTP requests — one as a normal browser, five using the actual published user-agents of ClaudeBot, GPTBot, OAI-SearchBot, PerplexityBot and Google-Extended — plus checks for robots.txt, llms.txt, llms-full.txt, sitemap.xml and security.txt. It's live-fetched from our own server (no domain sends CORS headers to a site it's never heard of), so there's no caching or guessing.

What it actually measures, in order of weight:

User-agent parity — status code and content length each crawler actually got back, versus what a browser got. Served / challenged / thin.
Text present without JavaScript execution
robots.txt at the root, per-agent, including Cloudflare's Content-Signal header
JSON-LD and prices declared in markup, not just painted on screen
Ratio of content tokens to total tokens inside <main>
Discovery files and basic semantic structure

Output is a 0-100 score with fixes ordered by points recoverable, not just a checklist.

I ran it against a few well-known sites: nytimes.com came back a D (58) — 4 out of 5 AI crawlers got a 403 straight from the edge, and robots.txt disallows everything. stripe.com scored A+ (93).

Worth being upfront: this space is crowded. There are a bunch of free robots.txt parsers out there, and crawlreadiness.com does something very close to this already (32 bots, real fetches, scored, paid tiers above a free daily limit). AgentReady is free, has no login, and the badge it generates is meant to sit on your own site — but I'm not claiming a moat here. What I actually think matters is the sibling project, The Agent Web Index: the same measurement, run continuously across the web, as a public index instead of a one-off check. That's the part nobody else is doing yet.

Curious what other checks people would want in the per-agent breakdown, or which crawlers I'm missing.

A free flight-price API that tells you buy or wait, not just today's fare

DEUS Automations — Tue, 15 Sep 2026 13:06:18 +0000

Every flight comparison site shows you today's cheapest fare. None of them will tell you to wait — it costs them the booking commission.

Piratefly watches flight prices over time and answers the actual question: is this price good, or should you wait? We just opened that logic up as a free JSON API.

The endpoint

GET https://next.piratefly.com/v1/analisi?origine=FCO&destinazione=BCN

Give it an IATA origin and destination, get back today's cheapest fare, where it sits against that route's own price history (percentile), and a buy/wait verdict.

curl "https://next.piratefly.com/v1/analisi?origine=FCO&destinazione=BCN"

Plans

Free — 30 requests/day per IP, no signup, no key.
Pro — $9.99/month — no daily cap. POST /v1/pro/checkout with your email, pay on Stripe, get an X-Api-Key back immediately.

Full machine-readable reference: https://next.piratefly.com/v1/docs
Landing page with the Pro checkout: https://next.piratefly.com/api

We only publish a verdict once a route has at least 8 price observations behind it — below that we say nothing rather than guess. Feedback and route requests welcome.

Your open-source repo's issue queue doesn't have to sit there for months

DEUS Automations — Wed, 09 Sep 2026 04:44:30 +0000

Small and medium open-source repos accumulate bug reports and feature requests that sit open for months because nobody has time to triage them. Every maintainer knows the feeling of opening the Issues tab and seeing 40+ items with zero labels, zero triage, zero idea which ones are quick fixes and which ones are rabbit holes.

GitHub Issue Triage BYOK is a small WordPress plugin that gives you a wp-admin dashboard listing a repository's open GitHub issues, with a one-click "Generate fix proposal" button. Click it and an AI model reads the issue title and body and gives you back a likely root cause, a concrete fix, and a suggested test — you copy, adapt, and act on it yourself.

A few things it deliberately does NOT do:

It never writes to GitHub, comments, labels, or opens pull requests. Read-only, always.
It doesn't ship an API key of its own. BYOK (Bring Your Own Key): you plug in your own GitHub token and your own AI key (any OpenAI-compatible chat completions endpoint works, not just OpenAI).
No subscription for the base functionality: the free version generates up to 20 AI fix proposals per month, which is enough to stay on top of a small/medium repo's queue. A one-time-payment Pro license removes the monthly cap if you need more.

Why WordPress and not a GitHub Action or a CLI tool? Because a lot of maintainers of small plugins/themes/tools already live in wp-admin for their own site, and having the triage dashboard one click away (rather than another terminal tab or CI log to check) is the difference between actually doing triage weekly and letting it pile up.

If you maintain a repo where issues outnumber your available time, it's on the WordPress plugin directory search and here: https://shop.lumnika.com/gh-issue-triage-byok.html

Happy to answer questions about the BYOK approach or how the fix-proposal prompt is built.

We Scanned Random Websites for Missing Alt Text. Even Wikipedia Failed.

DEUS Automations — Sat, 05 Sep 2026 02:47:50 +0000

Alt text isn't optional. Under WCAG 2.1 success criterion 1.1.1 ("Non-text Content"), every
meaningful image needs a text alternative — and it's one of the most commonly cited failures in
automated accessibility audits (WebAIM's Million report has flagged missing alt text as the
single most frequent error on the web for years running).

Finding every offending image by hand is the boring part, so we built a small crawler that walks
a site's public pages and flags every <img> with a missing or empty alt attribute. To sanity
check it, we pointed it at a handful of real, well-known domains instead of a synthetic test
page.

What it found

One of the sites we scanned was wikipedia.org. A single-page scan of the portal landing page —
the one that lets you pick a language — turned up an image with no alt attribute at all: the
Wikipedia wordmark logo itself (Wikipedia-logo-v2.png). One of the most-visited, most-audited
sites on the internet, and its own front door ships a logo image a screen reader has no way to
describe.

That's not a dig at Wikipedia specifically — it's the point. If a site with that much scrutiny can
ship an unlabeled image on its landing page, it happens everywhere, silently, on every site that
doesn't check for it systematically.

Why this keeps happening

A few patterns we see repeatedly when auditing real sites:

Logos and icons get skipped. Teams treat "obviously branding" images as exempt from needing alt text, but a screen reader user still needs to know what's there (or that it's purely decorative and should have alt="", not a missing attribute).
Alt text is added once, at launch, and never re-checked. New pages, new CMS templates, new marketing images ship without it because nothing in the pipeline catches the omission.
"Missing" and "empty" get conflated. A missing alt attribute is a hard WCAG failure. An empty alt="" on a genuinely decorative image is correct. Telling the two apart at scale is exactly the kind of repetitive checking that's easy to skip by hand and easy to automate.

Checking your own site

If you want to see this on your own domain rather than take our word for it, we put a free,
no-signup version of the same scanner online: it crawls up to 10 public pages, lists every image
with missing or empty alt text, and drafts a ready-to-paste alt text for each one using a vision
model.

Try the free scanner →

No login, no email required for the sample scan — paste a domain and see what it finds. The paid
version (CSV + report, up to 300 images) is there if you want the full site covered end to end,
but the free scan alone is enough to tell you whether it's worth doing.

Watching a Solana wallet without running your own RPC node

DEUS Automations — Thu, 03 Sep 2026 00:23:51 +0000

Watching a Solana wallet without running your own RPC node

If you want to know the moment a specific Solana wallet's balance moves — a treasury, a whale you're tracking, your own hot wallet for a security check — the "proper" way people reach for is running a dedicated indexer or paying for a webhook service like Helius. For a single wallet with modest polling needs, that's overkill.

Solana's public RPC (api.mainnet-beta.solana.com) is rate-limited but free, and it's enough to poll a wallet's SOL and SPL token balances on a schedule without an API key at all. Here's the pattern I used, in case you're about to build a similar watcher and don't want to pay for infra you don't need yet.

The loop

Poll every 5 minutes — getBalance for SOL, getTokenAccountsByOwner for SPL tokens, against the public RPC endpoint. No auth, no key.
Diff against last known balance — stored between runs (a simple key-value store is enough; I used n8n's built-in static data, but a Sheet or a Redis key works the same).
Threshold check — only fire an alert if the change exceeds a threshold you set. Without this, you get pinged on every dust transaction and start ignoring the channel within a day — the threshold is what makes the alert actually useful instead of noise.
Alert with context — a Telegram message with the delta and a link straight to the transaction on Solscan, so you don't have to go look it up.

Where the public RPC runs out

At low polling frequency (minutes, not seconds) and a handful of wallets, the public endpoint holds up fine. If you're watching dozens of wallets or need sub-minute latency, you'll hit rate limits and want to swap in a dedicated RPC (Helius, QuickNode, Triton) — the workflow is built so that's a one-field change, not a rewrite, since the RPC URL is a single credential.

If you want the built version

Packaged as an n8n workflow — polling, diffing, thresholding, and the Telegram alert with the Solscan link already wired up:

👉 https://shop.lumnika.com/solana-wallet-watcher.html?utm_source=devto&utm_medium=article&utm_campaign=solana-wallet-watcher

$14.99, one-time, no API key required to get started since it uses the public RPC by default.

Anyone else tracking wallets for security/ops reasons rather than trading? Curious what thresholds people settled on before the alerts stopped being noise.

Syncing a CRM with Odoo without paying for an iPaaS subscription

DEUS Automations — Thu, 03 Sep 2026 00:23:24 +0000

Syncing a CRM with Odoo without paying for an iPaaS subscription

Odoo is great as an ERP, but the moment sales wants to keep using HubSpot or Pipedrive as their CRM, you're stuck with a sync problem: contacts and orders need to exist correctly in both places, in both directions, without double-creating records or silently dropping updates.

The "just use Zapier/Make/Workato" answer works, but at CRM+ERP volume you're often paying $50-300/mo for what is, underneath, a scheduled poll and a webhook. I ended up building this as a plain n8n workflow instead, and wanted to share the pattern since it generalizes to basically any "sync system A with system B" problem, not just Odoo.

The two directions are not symmetric, and that matters

Odoo → CRM: a scheduled poll every 15 minutes, using Odoo's write_date field to pull only records changed since the last run (a delta sync, not a full table scan — this is the difference between a sync that stays fast at 50 records and one that stays fast at 50,000). Each changed res.partner or sale.order gets upserted into the CRM via its REST API.

CRM → Odoo: a webhook, because CRMs push changes in real time and there's no reason to wait 15 minutes for a lead update to land in Odoo. The webhook branches on whether the incoming record is new (create) or existing (update), matched by an external-ID field kept on both sides.

The asymmetry is the part people usually get wrong when they build this the first time: they either poll both directions (adds needless latency on the CRM side) or webhook both directions (Odoo doesn't push webhooks natively without extra modules). Poll the system that doesn't push, webhook the one that does.

Matching records without collisions

Both res.partner (contacts) and sale.order (orders) get an external-ID field synced on both sides at creation time, so updates always match the right record instead of relying on fuzzy matching (name + email, which breaks the moment someone has a typo or a duplicate).

If you want the built version

I packaged this as a 27-node n8n workflow — Odoo delta poll, CRM webhook with create/update branching, external-ID matching for both contacts and orders:

👉 https://shop.lumnika.com/crm-odoo-sync.html?utm_source=devto&utm_medium=article&utm_campaign=crm-odoo-sync

$49, one-time, works with HubSpot, Pipedrive, or any CRM with a REST webhook — you import it into your own n8n instance and point it at your credentials.

If you've built a two-way sync between other systems, I'd like to hear what matching strategy you used instead of external IDs — curious if anyone's had a cleaner approach.

Turning a folder of PDFs into a webhook that returns JSON

DEUS Automations — Thu, 03 Sep 2026 00:23:21 +0000

Turning a folder of PDFs into a webhook that returns JSON (no OCR infra to run)

If you've ever had to pull data out of PDFs at scale — invoices, ID scans, scanned forms, shipping manifests — you know the annoying part isn't the OCR itself, it's everything around it: standing up a service, handling the queueing, storing the output somewhere useful, and doing it all without paying for a $200/mo document-AI SaaS for what's often a low-volume internal tool.

I ended up solving this with a single n8n workflow instead of a custom service, and it turned out simpler than I expected, so here's the shape of it in case you're about to build the same thing from scratch.

The pipeline

Webhook in — POST a PDF URL (or upload) to a single endpoint.
OCR — the file goes to OCR.space, which has a genuinely usable free tier (25k requests/month, no card required). No self-hosted Tesseract, no GPU, no Docker image to maintain.
Text → structure — the raw OCR text gets parsed into a best-effort table: rows, columns, a JSON array, and a CSV export, using a mix of layout heuristics on the OCR bounding boxes.
Log (optional) — every extraction gets appended to a Google Sheet automatically, so you get a free audit trail without adding a database.
Response — the webhook returns plain text + structured JSON + CSV, so any app, script, or Zapier/Make alternative can call it directly.

The part worth calling out: because OCR.space is a hosted API, the whole thing runs with zero infrastructure of your own — n8n is the only thing you need to keep running, and if you already use n8n for other automations, this is just one more workflow, not a new service to babysit.

Where this breaks down

Layout-heavy PDFs (multi-column, dense tables with merged cells) will always need some manual review — no OCR pipeline gets that free. This is aimed at the 80% case: invoices, forms, receipts, single-column reports, where "close enough JSON" saves someone an afternoon of manual data entry.

If you'd rather not build it

I packaged the exact workflow — webhook, OCR.space call, structuring logic, optional Sheets logging — as a ready-to-import n8n template:

👉 https://shop.lumnika.com/pdf-ocr-extractor.html?utm_source=devto&utm_medium=article&utm_campaign=pdf-ocr

$19.99, one-time, JSON file you import into your own n8n instance — no subscription, no vendor lock-in, you can edit every node.

Curious if anyone here has fought with layout-heavy PDF extraction and found a heuristic that actually holds up — table detection is still the weakest link in every pipeline I've tried.

Is Your Shopify or WooCommerce Store an ADA/WCAG Lawsuit Risk? Free 2-Minute Check

DEUS Automations — Tue, 01 Sep 2026 11:34:02 +0000

Every year, ADA (Title III) web-accessibility lawsuits against US retailers keep climbing, and a huge share of them cite the same root cause: product images with no alt text. If you run a Shopify or WooCommerce store, that's usually not a design choice — it's just what happens when a catalog grows to hundreds or thousands of SKUs and nobody goes back to fill in alt attributes.

I wanted to know how exposed a "typical" store actually is, so I built a free scanner and ran it against a handful of real storefronts. Here's what I found, and the tool so you can check your own store in about two minutes.

Why missing alt text is a legal risk, not just an SEO nitpick

Screen readers rely on alt text to describe images to blind and low-vision shoppers. When a product image has no alt, or a generic one like IMG_4021.jpg, a screen reader either skips it or reads the filename — the shopper has no idea what they're looking at, can't tell products apart, and often can't complete checkout.

That's the exact pattern plaintiffs' firms look for before sending a demand letter: it's easy to prove (view source), easy to explain to a judge, and expensive to defend even when the claim doesn't go to trial. Settlements for small-to-mid retailers commonly land in the $5,000–$20,000 range — for something that's a content-population problem, not a code bug.

What the scan actually checks

I built ComplianceSnap to answer one narrow question fast: out of all the product images on your store, how many have no usable alt text?

You paste a store URL, it crawls the product pages, and it reports:

total product images found
how many are missing alt (or have placeholder/filename-only alt)
an estimated risk band based on catalog size and % missing (a 2,000-SKU store at 80% missing is a very different risk profile than a 40-SKU store at 10%)

It's read-only — it doesn't touch your store, theme, or admin, it just reads the public pages the same way a screen reader or a plaintiff's audit tool would.

What I found scanning real stores

Running it against a small sample of public Shopify/WooCommerce stores, the missing-alt rate ranged from under 10% (stores that clearly have an accessibility process) to over 70% (stores where alt text was never part of the product-upload workflow at all). The pattern held regardless of store size — it's a process gap, not a platform limitation. Both Shopify and WooCommerce support alt text natively; almost nobody fills it in by default.

If your scan comes back bad

The scan itself is free and doesn't require an account. If you want the missing alt text actually written (AI-generated, per image, reviewed before you publish), that's a paid step — the free scan is just there so you know whether it's worth doing before you spend anything.

Try it here: https://shop.lumnika.com/compliancesnap/?utm_source=devto&utm_medium=article&utm_campaign=ada-alt-text

Curious what other devs/store owners have seen on this — did anyone here actually get an ADA demand letter over image alt text, or is it mostly theoretical risk?

Why Most "AI Resume Builders" Produce PDFs That Fail ATS Parsers

DEUS Automations — Sat, 29 Aug 2026 15:56:15 +0000

If you've ever run your own resume through an ATS (Applicant Tracking System) parser and watched
your job title or bullet points come out garbled, you've hit a problem that has nothing to do with
your content and everything to do with how the PDF was built.

The root cause: PDF is a layout format, not a text format

A PDF doesn't store "a paragraph followed by another paragraph." It stores absolute-positioned
glyphs on a page. Most polished-looking resume templates get that layout from HTML+CSS rendered
through a headless browser (Puppeteer/Chromium), which is great for visual fidelity and terrible
for text order: multi-column layouts, sidebars, and floated elements often get flattened in
reading order that has nothing to do with visual order. An ATS parser reads the underlying text
stream top-to-bottom, left-to-right in insertion order — so a two-column template can silently
interleave your work history with your skills list, or drop your most recent job title into the
middle of a bullet point.

What actually fixes it

Two things matter more than template beauty:

Build the PDF with a low-level drawing API, not a browser. Libraries like pdfkit let you place text in the exact order you write it in code — single-column templates render sidebar accents as decoration (colored rectangles, icons) that never enter the text stream, so the parser only ever sees the real content, in the real order.
Always keep a resume-only fallback path. If you're generating resume content with an LLM (turning raw notes into structured bullet points, for example), the model call will eventually fail or time out. A parser that falls back to a deterministic, rule-based structuring of the same raw input — instead of failing the whole request — means the user still gets a correct, ATS-safe PDF even when the AI provider has a bad day.

A concrete example

I ended up building this into CV Forge, a small AI resume generator: raw notes
go in, an LLM (OpenAI-compatible, tested against Groq's Llama 3.3 70B) turns them into a structured
CV JSON, and pdfkit renders it with a single-column-first "Compact" template designed to stay
ATS-safe, alongside two more visual templates for when you're sending directly to a human. If the
AI call fails, a local parser produces the same structured JSON so the PDF still comes out
correctly formatted.

The lesson generalizes past resumes: any time you're generating a PDF that a machine will read
back, decide before picking a rendering approach whether text-stream order matters as much as
visual layout. If it does, skip the headless browser and reach for a library that lets you control
the text stream directly.

DEV Community: DEUS Automations

Your CDN has an opinion about AI crawlers, and it is not the one you set

What is being counted

Cloudflare, per crawler

The shape tells you whether a human decided

Why this is hard to see

Limits, stated first

Three PWA decisions that survived a real dining room

1. Never call skipWaiting() automatically

2. Scope realtime channels to the venue, not the table

3. Offline means "degrade to something usable", not "queue everything"

The pattern underneath

Most AI-crawler blocking doesn't happen in robots.txt

Most AI-crawler blocking doesn't happen in robots.txt

It depends on who answers for your domain

Check yours

Limits, stated by us first

Your site's robots.txt says AI crawlers are welcome. Your CDN might disagree.

A free flight-price API that tells you buy or wait, not just today's fare

The endpoint

Plans

Your open-source repo's issue queue doesn't have to sit there for months

We Scanned Random Websites for Missing Alt Text. Even Wikipedia Failed.

What it found

Why this keeps happening

Checking your own site

Watching a Solana wallet without running your own RPC node

Watching a Solana wallet without running your own RPC node

The loop

Where the public RPC runs out

If you want the built version

Syncing a CRM with Odoo without paying for an iPaaS subscription

Syncing a CRM with Odoo without paying for an iPaaS subscription

The two directions are not symmetric, and that matters

Matching records without collisions

If you want the built version

Turning a folder of PDFs into a webhook that returns JSON

Turning a folder of PDFs into a webhook that returns JSON (no OCR infra to run)

The pipeline

Where this breaks down

If you'd rather not build it

Is Your Shopify or WooCommerce Store an ADA/WCAG Lawsuit Risk? Free 2-Minute Check

Why missing alt text is a legal risk, not just an SEO nitpick

What the scan actually checks

What I found scanning real stores

If your scan comes back bad

Why Most "AI Resume Builders" Produce PDFs That Fail ATS Parsers

The root cause: PDF is a layout format, not a text format

What actually fixes it

A concrete example

1. Never call `skipWaiting()` automatically