Half my week is a visa agency in Bali. The other half is Apify Actors, and the newest one is not a scraper but a monitor: you name a few Threads accounts or brand keywords, give the watch a name, put it on a schedule, and every run hands you only the posts it has never handed you before. Same name, same memory, one row per new post.
That shape sounds like a scraper with a Set bolted on, and for about a week I believed it was. The thing that breaks the illusion is a question a scraper never has to ask: how far back does the page I am diffing against actually go?
Because that is the whole game. A monitor compares "what is on the page now" with "what I have already delivered". Anything that fell off the bottom of the page between two checks was never compared to anything. It is not late. It is gone.
All numbers below are from runs I made on 15 October 2026, build 0.1.29, plus one run from 14 October on build 0.1.27. No login, no cookies, no browser — these are the pages Threads serves to a visitor who is not signed in.
Three accounts, one window
I ran the monitor in preview mode (no watch name, nothing remembered) against three accounts, asking for up to 25 posts each. What came back:
| Account | Rows | Distinct threads | Oldest → newest row | Depth |
|---|---|---|---|---|
@cnn |
4 | 4 | 18:59:07 → 21:00:21 UTC, same day | 2.02 hours |
@mosseri |
5 | 4 | 2026-10-02 13:32 → 2026-10-12 17:09 | 10.15 days |
@zuck |
10 | 4 | 2026-09-25 16:50 → 2026-10-12 16:56 | 17.0 days |
The row counts are 4, 5 and 10. The thread counts are 4, 4 and 4.
The profile HTML does not embed a number of posts. It embeds four threads, and a thread is one post or a chain of them. @zuck's four threads happen to be 4 + 1 + 4 + 1 posts:
2026-10-12T16:56:29Z thread=4006434140014576097 pos=0/4
2026-10-12T16:57:09Z thread=4006434140014576097 pos=1/4
2026-10-12T16:56:28Z thread=4006434140014576097 pos=2/4
2026-10-12T16:56:28Z thread=4006434140014576097 pos=3/4
2026-10-12T15:01:17Z thread=4006376160339141718 pos=0/1
2026-09-28T12:35:32Z thread=3996155940894885511 pos=0/4
…
@cnn's four threads are four single posts, which is why a news desk that posts all day gives you a two-hour window and a Meta executive who posts twice a week gives you seventeen days. Same markup, same four slots, a 200× difference in what they cover.
Your schedule is in minutes; the window is in threads
This is the part I had wrong when I wrote the first version. I had reasoned about the check interval in the only unit I controlled — minutes — and picked an hour, because most Threads accounts post a few times a day and an hourly check is plainly enough for them.
It is plainly enough for @mosseri. For @cnn the same hour buys you a window with two slots of headroom: the account was running at roughly two posts an hour, four threads deep. A check every three hours against the embedded page alone would have silently skipped posts, and the monitor would have reported nothing wrong, because every post it did see was correctly marked new. Silent, plausible, wrong — the worst failure a scheduled job can have.
The conversion factor between "four threads" and "how long I may sleep" is the account's posting rate, and that is the one variable in the system that belongs to someone else.
The signal that you fell off the end
You cannot know the posts you did not see. But there is a tell, and it is cheap: if every single thread on the page is new to the watch, the page no longer overlaps your memory, and an account that posted exactly four times since the last check is far less likely than an account that posted more.
So the normal check reads one page. Only when the overlap is empty does it ask for the next page of the profile feed:
const gap = !isPreview && !isFirstCheck && rows.length
&& rows.filter((r) => !r.isPinned).every((r) => !seenBefore.has(String(r.id || '')));
if (gap && rows.length < limit && tab?.page_info?.has_next_page && cursor) {
// one refetch, 25 posts, stop at the first id this watch already delivered
}
Seeded a watch with a memory that deliberately did not overlap and pointed it at @cnn:
INFO Watch "devto-gap2": 1 post(s) already delivered by earlier runs.
INFO @cnn: 10 new of 25 checked
INFO Done: {"checked":1,"newPosts":10,"pages":2,"requests":2,"bytes":1309270}
Two requests instead of one, 25 posts instead of 4. The ten rows it delivered (my maxItems cap) span 5.65 hours; the 25 it read reach back about fourteen hours at that posting rate. A quiet account never triggers this and keeps paying one request per check — which matters when a schedule fires 720 times a month.
Why the cursor is an id and never a timestamp
Look again at @zuck's four rows in feed order:
16:56:29 16:57:09 16:56:28 16:56:28
Not descending. Not ascending. The feed is newest-thread-first, but a thread's own posts come out in reply order, and the author wrote the second link of that chain forty seconds after the first and then back-filled. Store "the newest createdAt I have delivered" as your cursor — the obvious design, the one every incremental-sync tutorial shows you — and the watermark lands at 16:57:09. The two posts at 16:56:28 that follow it in the feed are then older than the watermark, so they are dropped. Two of ten posts lost on a page where nothing went wrong.
The memory is therefore a set of post ids, and the gap scan stops at the first id already in that set rather than at a time. The same rule makes the second page safe: everything after the first familiar id is older than the last check and must not come back as new.
One consequence worth stating plainly: filtered-out posts go into that set too. A post that failed your minLikes floor today has been seen, and should not arrive tomorrow as breaking news because it finally got its tenth like.
The cheapest useful thing a monitor does is refuse to look
A watch will not check the same target more than once an hour. Fire it sooner and it stops before any request:
Watch "devto-demo-15oct" was checked 29 s ago. A watch is checked at most once an hour,
so this run did not look again. Nothing is lost — anything new arrives with the next check.
That run finished in 1.245 seconds, made zero HTTP requests and delivered zero rows. Given the four-thread window, a tighter interval is not obviously wrong — it is what a crisis watch wants — but it has to be a decision, not an accident of someone copying a */5 * * * * cron line.
Calling it
Named watch, two targets, one schedule — the whole thing:
curl -X POST "https://api.apify.com/v2/acts/lergassy~threads-monitor/runs?token=$APIFY_TOKEN" \
-H 'Content-Type: application/json' \
-d '{"watchName":"brand-mentions","usernames":["cnn"],"keywords":["my brand"],"maxPostsPerCheck":25}'
Reuse watchName and every later run returns the difference. Leave it empty and you get a preview that remembers nothing, which is how the three measurements above were taken: Threads Monitor.
When this is the wrong tool
- You want history. A monitor's answer to "what did they post in June" is nothing at all. That is a Threads Scraper job — search, full accounts, whole threads, profiles.
- You want sub-minute latency. Four threads deep and a once-an-hour floor is not a firehose, and public pages will never be one.
- You want your own account's numbers. Reach, impressions, follower demographics — that is the official API, which exists precisely and only for the account you own.
- You already have a dedup table you trust. Then you have a monitor; it is just made of a scraper plus your table, and you are paying for the overlap you throw away.
And do not hardcode four. It is a server-side rendering choice that owes me nothing: three accounts, one day, one shape. The heuristic that says "no overlap means read further" survives Meta changing that number. A constant does not.
Top comments (0)