DEV Community: Daniel Root

Let an AI agent set up uptime and change alerts with one API call

Daniel Root — Wed, 30 Sep 2026 14:10:47 +0000

AI agents are good at fetching a page once and reasoning about it. They are less good at the boring middle: a durable schedule, a paid subscription, and a signed delivery channel that keeps working after the chat ends. That is the gap we built PingWhen's agent API for.

If your agent can read a short doc, POST JSON, and hand a human a checkout link, it can set up page-change and uptime alerts without scraping, cron, or a free-tier dashboard.

What the agent needs to know first

Two public docs are enough:

https://pingwhen.app/llms.txt — short product summary for LLM loaders
https://pingwhen.app/for-agents — the create / poll / webhook / cancel flow
OpenAPI at https://pingwhen.app/v1/openapi.json if you want a machine-readable schema

There is no free plan and no agent-only price. Agents pay the same Stripe subscription as humans: Starter $9/mo (10 watches, every 15 min) or Plus $19/mo (40 watches, every 5 min).

The flow in five steps

Read /llms.txt or /for-agents.
POST /v1/watches with the URL to watch and where to send alerts.
Give the human the returned checkout_url (or open it in an approved browser tool).
Poll GET /v1/watches/:id?token=... until status is active.
Receive signed webhook POSTs when the page changes, goes down, or comes back.

Payment is required before polling starts. The create call inserts the watch as pending_payment, opens Stripe Checkout, and returns immediately.

Create a watch

curl -sS -X POST https://pingwhen.app/v1/watches \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://example.com/status",
    "notify": "https://agent.example/hooks/pingwhen",
    "events": ["change", "down", "up"],
    "plan": "starter",
    "success_url": "https://agent.example/paid",
    "cancel_url": "https://agent.example/abandoned"
  }'

notify may be an HTTPS webhook URL (preferred for agents), an email, or a US/CA phone number. events is any subset of change, down, and up. plan is starter or plus. success_url and cancel_url are optional Stripe return URLs.

A typical 201 response looks like:

{
  "watch_id": "w_...",
  "status": "pending_payment",
  "manage_token": "mt_...",
  "plan": "starter",
  "checkout_url": "https://checkout.stripe.com/c/pay/cs_...",
  "poll": "GET /v1/watches/w_...?token=mt_...",
  "cancel": "DELETE /v1/watches/w_...?token=mt_...",
  "instructions": "Open checkout_url in a browser. Then poll until status=active."
}

Store manage_token. You need it to poll, cancel, and verify webhook signatures. If notify is an email for an existing customer who still has spare quota, Checkout is skipped and status comes back active right away.

Poll until active

curl -sS "https://pingwhen.app/v1/watches/w_...?token=mt_..."

Status values: pending_payment | active | past_due | canceled | expired. Once active, the JSON includes a Stripe Customer Portal URL so the human can manage billing.

Verify webhook signatures

Alerts to an HTTPS notify URL are JSON POSTs:

{
  "event": "change",
  "watch_id": "w_...",
  "url": "https://example.com/status",
  "checked_at": "2026-09-20T20:04:00Z",
  "http_status": 200,
  "excerpt": "text now includes $19/mo"
}

event is one of change | down | up | active. Headers:

User-Agent: PingWhenBot/1.0
X-PingWhen-Signature: sha256=<hex>
X-PingWhen-Event: change (same as the body event)

The signature is HMAC-SHA256 of the raw request body, using manage_token as the secret. Header format is exactly sha256= plus the hex digest.

Node:

const crypto = require("crypto");

function verifyPingWhen(rawBody, signatureHeader, manageToken) {
  const expected =
    "sha256=" +
    crypto.createHmac("sha256", manageToken).update(rawBody).digest("hex");
  const a = Buffer.from(expected);
  const b = Buffer.from(signatureHeader || "");
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

Python:

import hashlib, hmac

def verify_pingwhen(raw_body: bytes, signature_header: str, manage_token: str) -> bool:
    digest = hmac.new(manage_token.encode(), raw_body, hashlib.sha256).hexdigest()
    expected = "sha256=" + digest
    return hmac.compare_digest(expected, signature_header or "")

Use the raw bytes you received, not a re-serialized JSON object. Respond with 410 Gone (or {"ok":false,"unsubscribe":true}) to delete the watch. We retry failed deliveries a couple of times before giving up.

Honest limits (read these before you wire it in)

We watch the HTML we receive: visible text after scripts/styles are stripped. We do not run a browser or execute JavaScript. If the words you care about are missing from view-source, use a JS-capable monitor or watch the JSON API the page loads instead.
No free plan. Starter is $9/mo for 10 watches every 15 minutes; Plus is $19/mo for 40 watches every 5 minutes.
Watch URLs go through SSRF protections (public HTTP(S) only). Private/local targets are refused.
The first check is a silent baseline. You get alerted on later changes, not on the initial snapshot.
Webhook destinations must be HTTPS. Email and US/CA SMS also work if that fits the human better than a hook.

Not sure whether a URL is a good candidate? The free one-shot page check fetches it the same way our watcher would and shows status, size, and a text preview.

Where this fits in an agent loop

A practical pattern: the human asks the agent to "watch this status page and ping my webhook if it dies or the copy changes." The agent reads /for-agents, creates the watch, returns the checkout link, polls to confirm payment, and then treats signed webhooks as durable events. Cancel a single watch with DELETE /v1/watches/:id?token=...; cancel the subscription through the portal URL once the watch is active.

We built PingWhen for the simple case — public, server-rendered pages and JSON URLs — not as a general browser automation platform. If that matches what your agent needs, start at /for-agents or the homepage at pingwhen.app.

Originally published on the PingWhen blog. Written with AI assistance for PingWhen, the page-change and uptime alert tool we make.

Monitor your robots.txt and sitemap for accidental changes

Daniel Root — Sat, 26 Sep 2026 20:21:15 +0000

Two SEO accidents come up again and again:

A staging robots.txt with Disallow: / gets deployed to production.
A migration or plugin update changes the sitemap generator, and whole sections disappear from sitemap.xml.

Neither breaks the site for visitors, so nobody notices. Search engines do notice, just slowly: Google says it generally caches robots.txt for up to 24 hours, and the effect on indexing and traffic can take much longer to show up in reports. By the time someone asks "why is organic down?", the cause is weeks old.

The fix is boring: watch the few files that control crawling, and add a smoke test to your deploys.

What to watch

/robots.txt: plain text, small, and should almost never change. Any change is worth a human look.
Your sitemap index and the child sitemaps that matter. If your sitemap is split (sitemap-pages.xml, sitemap-products.xml...), watch the index plus the ones for your most important sections.
The visible text of 3-5 money pages (home, pricing, top landing pages), which catches a template change that wipes copy.
Other plain-text control files you rely on, such as /ads.txt or /.well-known/security.txt.

What a text-change monitor will not catch

Most change monitors, ours included, compare a page's visible text. That's great for robots.txt and sitemaps, but several indexing controls live in markup or headers, not in text:

<meta name="robots" content="noindex"> (an attribute, not visible text)
the X-Robots-Tag: noindex HTTP header
rel="canonical" and hreflang changes
tags injected by JavaScript after load

For those, use Search Console's indexing reports, a scheduled crawl (Screaming Frog, Sitebulb and similar tools can run on a schedule), or better, a check that runs on every deploy.

A post-deploy SEO smoke test

Drop this into your CI after deploy (or run it from cron). It exits non-zero if production looks de-indexed:

#!/usr/bin/env bash
# seo-smoke.sh: fail loudly if production is telling crawlers to go away
set -uo pipefail
site="https://example.com"
pages=(/ /pricing /blog)
fail=0

robots=$(curl -fsS "$site/robots.txt") || { echo "robots.txt not reachable"; fail=1; }
if grep -Eiq '^[[:space:]]*disallow:[[:space:]]*/[[:space:]]*$' <<<"$robots"; then
  echo "robots.txt contains 'Disallow: /'. Check which user-agent group it's in"
  fail=1
fi

for path in "${pages[@]}"; do
  headers=$(curl -fsSI "$site$path") || { echo "$path: not reachable"; fail=1; continue; }
  body=$(curl -fsS "$site$path")
  if grep -qi '^x-robots-tag:.*noindex' <<<"$headers"; then
    echo "$path: X-Robots-Tag noindex"; fail=1
  fi
  if grep -Eiq '<meta[^>]*name=.?robots[^>]*noindex' <<<"$body"; then
    echo "$path: meta robots noindex"; fail=1
  fi
done

count=$(curl -fsS "$site/sitemap.xml" | grep -o '<loc>' | wc -l)
echo "sitemap.xml lists $count URLs"
if [ "$count" -lt 10 ]; then echo "sitemap looks too small"; fail=1; fi

exit $fail

Adjust before trusting it:

The robots check flags Disallow: / anywhere in the file, including groups for specific bots you meant to block. Treat a hit as "look at this", not proof.
The meta check assumes name comes before content, which is the common order but not guaranteed.
Set the sitemap threshold to something meaningful for your site, e.g. 80% of last week's count. If you use a sitemap index, count the child sitemaps instead.

Between deploys: change alerts

The smoke test covers your deploys. It doesn't cover a CMS plugin update at 2am, a teammate editing robots.txt in an admin panel, or a hosting platform "helpfully" changing defaults. That's where a change monitor on the files above earns its keep.

Some practical notes for sitemaps specifically:

If your generator rewrites <lastmod> on every build, a text monitor will alert on every deploy. Full ISO timestamps like 2026-09-26T10:00:00+00:00 are ignored by monitors that mask timestamps (PingWhen does), but date-only values like 2026-09-26 are not. Watch a child sitemap that changes rarely, or accept a deploy-time alert.
Very large sitemaps (tens of thousands of URLs) are better handled by the count check above than by a diff.

With PingWhen

PingWhen watches public URLs and emails you, or POSTs a signed JSON webhook, when their text changes, when they go down, and when they come back. Because robots.txt is plain text, every edit produces an alert with a short excerpt of the added text. Point the webhook at your CI (anything that can take a JSON POST) and you can re-run the smoke test automatically whenever robots.txt or a sitemap changes.

It's $9/month for 10 URLs checked every 15 minutes, and there's no free plan. You can check whether your files are reachable and readable with our free page check.

Originally published on the PingWhen blog. Written with AI assistance for PingWhen, the page-change and uptime alert tool we make.

How to get notified when a web page changes (without writing a scraper)

Daniel Root — Sat, 26 Sep 2026 20:21:11 +0000

You want to know when a page changes: a "registration opens soon" notice, a competitor's pricing table, a changelog with no RSS, a government page that says "updates will be posted here". Most guides jump straight to a tool. It's worth spending five minutes first, because the right URL matters more than the right tool.

Step 0: check whether a feed already exists

A feed is the most reliable change notification there is, and it's free.

Look in the page source for <link rel="alternate" type="application/rss+xml"> or application/atom+xml.
Try the usual paths: /feed, /rss, /atom.xml, /index.xml.
GitHub has Atom feeds for releases (https://github.com/OWNER/REPO/releases.atom), tags (/tags.atom) and commits (/commits/main.atom).
Subreddits have one at https://www.reddit.com/r/NAME/.rss.

If there's a feed, put it in any feed reader (or an automation tool that emails you new items) and stop here.

Step 1: the view-source test

Open the page, then open its source: Ctrl+U / Cmd+Option+U, or prefix the address with view-source:. Search for the words you care about.

The text is in the source. The page is server-rendered. Any text-based monitor will see what you see, and a simple, cheap tool is enough.
The source is mostly <script> tags and an empty <div id="root">. The page is rendered by JavaScript in the browser. A plain HTTP fetch sees an empty shell.

For JavaScript pages you have two options:

Find the data URL. Open DevTools, go to the Network tab, filter by Fetch/XHR, and reload. The page is usually loading JSON from an API. If that JSON URL works when you open it in a new tab (no login cookie needed), watch the JSON instead of the page. It's smaller, it changes only when the data changes, and many monitors handle JSON well.
Use a monitor that runs a real browser (Visualping, PageCrawl, or self-hosted changedetection.io with its Playwright fetcher). These are heavier and usually cost more per check, but they work on single-page apps.

Step 2: pick the narrowest URL you can

Whatever tool you use, noise usually comes from the page, not the tool. Before you set anything up:

Prefer the specific page (one product, one notice, one job posting) over a homepage or listing.
Look for a print view, a "text only" version, or a filtered URL (?category=...) that shows only what you need.
Watch out for text that changes on every load: "posted 5 minutes ago", rotating testimonials, view counters, "people are looking at this now". A monitor that compares the whole page will alert on every one of those. Either pick a different URL or use a tool that lets you select a region.
Markup-only churn (cache-busting file names, CSRF tokens in attributes, analytics snippets) is harmless for monitors that compare visible text. It's a problem for ones that compare raw HTML.

Step 3: choose how you want to be alerted

There are three broad families. None is "best"; they trade money for effort differently.

Approach	Examples	Good at	Watch out for
Browser extension, local checks	Distill (local monitors)	Free, can see pages you're logged into	Only checks while your browser is running
Self-hosted	urlwatch, changedetection.io	Free and flexible (selectors, filters, JS rendering)	You run and update it, and it's down when your box is
Hosted service	Visualping, PageCrawl, ChangeTower, PingWhen	Checks while your laptop is closed, no maintenance	Monthly cost; free tiers are usually small

If you're comfortable with a terminal, the smallest possible self-hosted version is a cron job. This one hashes the page's text and emails you when the hash changes:

#!/usr/bin/env bash
# watch.sh URL: run from cron, e.g. */15 * * * * /home/me/watch.sh https://example.com/notice
set -euo pipefail
url="$1"
state="$HOME/.pagewatch/$(printf '%s' "$url" | sha256sum | cut -c1-16)"
mkdir -p "$(dirname "$state")"

new=$(curl -fsSL --max-time 20 -A 'pagewatch/1.0' "$url" \
  | sed -e 's/<script[^>]*>.*<\/script>//g' -e 's/<[^>]*>/ /g' \
  | tr -s '[:space:]' ' ' \
  | sha256sum | cut -d' ' -f1)

old=$(cat "$state" 2>/dev/null || true)
if [ -n "$old" ] && [ "$new" != "$old" ]; then
  echo "$url changed" | mail -s "Page changed: $url" you@example.com
fi
echo "$new" > "$state"

Honest caveats: the sed is crude (a <script> block that spans several lines survives it), it needs a working mail command, and because of curl -f it silently skips a run when the site is down instead of telling you. It's a fine starting point for one or two pages. Past that, a real tool pays for itself in time.

Step 4: expect a baseline, then tune

Every monitor has to store a first copy before it can tell you anything changed, so the first check is silent. If you then get alerts that don't matter, go back to Step 2 before you blame the tool. A narrower URL fixes most noise.

Where PingWhen fits

We built PingWhen for the simple case: a public, server-rendered page (or a JSON URL) where you just want an email or a webhook when the text changes, plus a "down" and "back up" alert on the same URL. It compares the page's visible text (scripts and styles removed, ISO timestamps masked, JSON keys sorted) and sends one alert with a short excerpt of what's new. It doesn't render JavaScript or let you pick a CSS selector, and there's no free plan: it's $9/month for 10 pages checked every 15 minutes.

Not sure whether a page is a good candidate? Our free page check fetches it once, the way our watcher would, and tells you whether it's up and whether we can see its text.

Originally published on the PingWhen blog. Written with AI assistance for PingWhen, the page-change and uptime alert tool we make.