DEV Community

Nazma Akter
Nazma Akter

Posted on

One inbox for 8 networks: polling comments without burning API quota

#webdev #go #postgres #showdev

VeloxonHub Social has an Inbox that collects comments on your published posts from Facebook, Instagram, YouTube, Threads, Bluesky, Mastodon, DEV.to and WordPress, and lets you reply from one screen. Most of these networks don't push comments to you, so the Inbox is built on polling. Here's how we keep that cheap, correct and threaded.

Poll by post age, not on a fixed clock

Comments arrive fast while a post is new and almost stop after a couple of days. Polling every post every 15 minutes wastes quota: YouTube gives the whole app about 10,000 units a day, so a 15-minute comment poll for 14 days would cap us at roughly 100 videos.

So each published post has a cadence:

inboxCadence = repository.PollCadence{
    FreshFor: 2 * 24 * time.Hour, Fresh: 15 * time.Minute,
    Stale:    3 * time.Hour,      MaxAge: 14 * 24 * time.Hour,
}
Enter fullscreen mode Exit fullscreen mode

Every 15 minutes for the first 2 days, then every 3 hours, and never after 14 days. The "is this post due?" check is a single SQL expression, so the worker asks Postgres for the next batch instead of keeping timers:

AND pt.published_at > now() - $2::interval            -- MaxAge
AND (pt.inbox_checked_at IS NULL OR pt.inbox_checked_at <
     now() - CASE WHEN pt.published_at > now() - $3::interval  -- FreshFor
                  THEN $4::interval ELSE $5::interval END)     -- Fresh / Stale
ORDER BY pt.inbox_checked_at NULLS FIRST
Enter fullscreen mode Exit fullscreen mode

Posts that were never checked go first, and only posts on active accounts are polled.

Inserts that keep read state

Each poll returns the comments the network currently has, most of which we've already seen. They're saved with:

ON CONFLICT (social_account_id, external_id) DO NOTHING
Enter fullscreen mode Exit fullscreen mode

An existing comment keeps its read and replied state, and RowsAffected() tells us exactly which comments are new. Only those trigger the bell notification (and a push alert in our iOS and Android apps, which are coming soon), so a re-poll never notifies twice.

Threads from one extra column

Networks return replies to comments as well as top-level comments. Storing the parent's external id (parent_external_id) is enough to build threads. The frontend groups the flat list into comment → reply → reply trees, sorts threads by their newest activity, and shows a reply on its own (with "Replying to …") when its parent is filtered out.

Rows saved before we tracked parents fill in on their next poll: if the insert hits a conflict and the network now reports a parent, we update just that column.

Replies: one interface per capability

Not every network lets an app reply, and fewer allow images in replies. Each network's client registers what it can do: a Replier, and optionally an ImageReplier. The API returns can_reply and can_reply_image per comment, so the UI only shows what will actually work. Where replies aren't possible, the UI says so and links to the comment on the network.

Before sending we check the network's character limit, and that an attached image is an image from the workspace's own media library. If the network rejects the reply, the user sees the network's own reason, plus "reconnect the account to grant reply permission" when it was a permissions error.

What we'd tell you if you're building something similar

  • Poll by age. Most comments arrive in the first two days.
  • Let the database decide what's due. One indexed query beats a scheduler per post.
  • Use ON CONFLICT DO NOTHING and count affected rows to find what's new.
  • Model capabilities per network, and expose them to the UI.

The Inbox is on every VeloxonHub Social plan, with a 7-day free trial: https://social.veloxonhub.com. Happy to go deeper on the per-network clients in the comments.

Top comments (0)