DEV Community: Signal Wharf

How to Get Substack Newsletter Data: Posts, Comments and Subscriber Counts

Signal Wharf — Wed, 11 Nov 2026 23:52:09 +0000

Substack hosts many of the most-read independent newsletters, and the data around them is useful. Sponsors want to know how big a newsletter really is. Researchers want the full text of posts. Media teams track what competitors publish and how readers react.

Substack has no official public API for this. The data is visible in the browser, though: post pages, comment threads, the archive, and the "Over 466,000 subscribers" line on a publication's homepage. This guide shows how to collect it in a structured way without logging in, using the Substack Scraper on Apify. The output works as JSON, CSV or Excel, or from Python.

What data you can get from Substack

Posts (one item per post):

title, subtitle, URL, publish date, authors, section and tags
isPaid and audience (free or subscriber-only)
word count, likes (reactions), comment count and restacks
full body text, optionally the HTML, and the links in the post
podcast URL and duration for audio posts

Publication profiles (one per newsletter):

the subscriber estimate as shown publicly ("Over 466,000 subscribers" → 466000)
the paid-subscriber tier text, and the Substack bestseller tier
monthly, yearly and founding-member prices
posting cadence (posts per week), the share of posts that are paid, and average likes, comments and restacks

Comments (optional): author, date, body, likes, reply count and thread depth.

What it does not do: it doesn't log in, bypass paywalls or collect subscriber emails. For subscriber-only posts you get the metadata plus the free preview that Substack shows every visitor, flagged with isFullTextComplete: false. Substack Notes aren't covered.

Step 1: List the newsletters

Use whatever identifier you have. The scraper accepts:

a subdomain: noahpinion
a Substack URL: https://noahpinion.substack.com
a custom domain: https://www.astralcodexten.com or www.noahpinion.blog
a profile link: https://substack.com/@noahpinion

Step 2: Run the scraper

Open Substack Scraper and click Try for free. You need a free Apify account.
Add your newsletters under Publications.
Set Max posts per publication. Use 20–50 for a quick look, or more for a full archive.
Keep Include full text on if you want the article bodies.
Turn on Include comments if you need them. You can also set Max comments per post.
Optional: add a date range or keywords.
Click Start.

Our test input (Nov 11, 2026):

{
  "publications": ["noahpinion", "https://www.astralcodexten.com"],
  "maxPostsPerPublication": 20,
  "includeFullText": true,
  "includePublicationProfile": true
}

It finished with 40 posts from 2 of 2 publications, plus a profile for each.

Step 3: Read the output

Every item has an itemType: post, publication or comment. Here are real items from our test runs, trimmed.

A publication profile:

{
  "itemType": "publication",
  "publicationName": "Noahpinion",
  "publicationUrl": "https://www.noahpinion.blog",
  "authorName": "Noah Smith",
  "createdAt": "2020-03-28T03:32:51.086Z",
  "subscriberCountText": "Over 466,000 subscribers",
  "subscriberCountApprox": 466000,
  "paidSubscriberTierText": "Tens of thousands of paid subscribers",
  "bestsellerTier": 10000,
  "monthlyPrice": 10.0,
  "yearlyPrice": 100.0,
  "foundingMemberPrice": 420.0,
  "postsPerWeek": 7.0,
  "avgReactions": 190.3,
  "avgComments": 12.7
}

A post:

{
  "itemType": "post",
  "publicationName": "Noahpinion",
  "title": "India is quietly doing great",
  "subtitle": "The world’s biggest country continues to defy the skeptics.",
  "url": "https://www.noahpinion.blog/p/india-is-quietly-doing-great",
  "postDate": "2026-11-11T08:51:56.921Z",
  "audience": "everyone",
  "isPaid": false,
  "wordcount": 1596,
  "reactionCount": 404,
  "commentCount": 25,
  "restacks": 47,
  "bodyText": "My last post was about how for the first time, I’m becoming bearish on China’s ..."
}

A comment (with comments enabled):

{
  "itemType": "comment",
  "postTitle": "India is quietly doing great",
  "authorName": "Matthew",
  "date": "2026-11-11T09:29:04.810Z",
  "reactionCount": 35,
  "replyCount": 3,
  "depth": 0,
  "body": "Great piece. I wish you had gone into some of the politics of this, ..."
}

How accurate are the subscriber counts?

They're as accurate as what Substack shows publicly, and that's a rounded figure. "Over 466,000 subscribers" becomes subscriberCountApprox: 466000. The real number is at least that high, and it includes free subscribers. Paid subscribers only appear as a tier ("Tens of thousands of paid subscribers"), never as an exact count. Some newsletters hide the count entirely, and then the field is null. The averages (avgReactions, postsPerWeek) are computed from the posts scraped in that run. The field statsBasedOnPosts tells you how many posts that was, so scrape more posts if you want steadier averages.

Step 4: Export to CSV, Excel or Google Sheets

The Output tab has views for each item type:

View	Best for
Overview	one row per post: date, title, paid flag, word count, likes, comments, restacks
Content	full text, authors, tags and links, for reading or analysis
Publications	subscriber estimates, prices, bestseller tier, cadence and engagement
Comments	comment text, author, likes and depth

Choose a view, then Export as CSV, Excel or JSON. To send results to Google Sheets automatically, use Apify's Google Sheets integration on the Actor or on a scheduled task.

Step 5: Use it from Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("signalwharf/substack-scraper").call(run_input={
    "publications": ["noahpinion", "https://www.astralcodexten.com"],
    "maxPostsPerPublication": 20,
})

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
pubs = [i for i in items if i["itemType"] == "publication"]
posts = [i for i in items if i["itemType"] == "post"]

for p in pubs:
    print(p["publicationName"], p["subscriberCountApprox"], p["monthlyPrice"])

# Most-liked posts first
for post in sorted(posts, key=lambda x: x["reactionCount"] or 0, reverse=True)[:5]:
    print(post["reactionCount"], post["title"])

With pandas, pd.DataFrame(posts) gives you a table ready to filter or plot.

Step 6: Track new posts automatically

Set onlyNewPosts: true and create a schedule, for example hourly or daily. The scraper remembers the newest post it saw for each publication and returns only newer ones on the next run. A run that finds nothing new costs nothing. This is a simple way to build a newsletter monitor that feeds Slack, email or a spreadsheet.

Practical use cases

Sponsorship research: compare subscriber estimates, posting cadence and engagement across newsletters before you buy an ad slot.
Competitive content tracking: see what competitors publish, how long their posts are, and which topics get the most likes and restacks.
Research and NLP datasets: collect full text from free posts for topic modeling, summarization or sentiment analysis.
Pricing benchmarks: see the monthly and yearly prices that newsletters in your niche charge.

How much does it cost?

Pay per event, no subscription:

Event	Price
Post (full text included)	$0.0015 ($1.50 per 1,000)
Publication profile	$0.003
Comment (only if enabled)	$0.0003

Our 40-post test with 2 profiles cost 40 × $0.0015 + 2 × $0.003 = $0.066. A single newsletter with 100 posts and a profile costs about $0.15. You can cap the cost per run in the run options. Apify's free plan includes a monthly platform credit, which is enough to try it out. Check Apify's pricing page for the current amount.

Troubleshooting

A custom domain isn't recognized: paste the full homepage URL, e.g. https://www.astralcodexten.com.
Paid post with short text: expected. Only the public preview is available without a subscription.
Subscriber fields are null: that publication doesn't show a public count.
Comment counts differ from the comments returned: check maxCommentsPerPost.

Is it legal?

The scraper reads only public pages and public JSON that any visitor's browser loads. It doesn't log in, bypass paywalls, or collect emails or private subscriber data. Post text is copyrighted by its authors, so use it for analysis and research, and don't republish it. Comments are personal data under the GDPR. Ask a lawyer if you're unsure.

Related tools

Telegram Channel Scraper: public Telegram channel posts, views and reactions, no login.
AI Brand Visibility Tracker: check whether ChatGPT, Claude and Gemini recommend your brand.

Built and maintained by Signal Wharf. Substack is a trademark of its owner; this tool is not affiliated with or endorsed by Substack. Example newsletters are shown only to illustrate public data.

How to Scrape a Telegram Channel to CSV (No Login)

Signal Wharf — Wed, 11 Nov 2026 23:49:52 +0000

Public Telegram channels are a good source of news, crypto signals, product announcements and community chatter. Getting that data into a spreadsheet is harder than it should be. The official Telegram API (MTProto) needs an account, a phone number and an api_id. Libraries like Telethon need a logged-in session, and copying posts by hand doesn't scale.

There is a simpler route for public channels. Telegram publishes a web preview of every public channel at https://t.me/s/<channel>. Anyone can see it without logging in. This tutorial shows how to turn that preview into a clean CSV: posts, view counts, reactions, links and media. It uses the Telegram Channel Scraper on Apify. You can run it from the browser, or call it from Python or curl.

What you can and can't scrape

What you can get from a public channel:

post text, date (UTC), post URL and ID
view count and the reaction breakdown (including paid ⭐ Stars reactions)
links, hashtags, mentions and link previews
photos, videos and documents (CDN URLs), forwards and replies
channel title and subscriber count

What you can't get, by design: private channels, invite links (t.me/+...), groups, chats, comment threads and member lists. If you need those, you need the official API and an account. Some channels also switch off the web preview. The scraper reports these as preview_disabled and doesn't charge for them.

Step 1: Check that the channel has a web preview

Open https://t.me/s/durov in a browser. If you see a scrollable list of posts, the channel can be scraped this way. If Telegram only shows a "View in Telegram" button, the owner has disabled the preview.

The input accepts several formats, so use whichever you have:

durov
@durov
https://t.me/durov
https://t.me/s/durov

Step 2: Run the scraper

Open Telegram Channel Scraper and click Try for free. You need a free Apify account. No credit card is needed for the free plan.
In Channels, add one or more channel names.
Set Max posts per channel. Start with 100. Telegram serves about 20 posts per page, so 100 posts is about five page loads.
Optional: set Date from / Date to for a time window, or add Keywords to keep only matching posts.
Click Start.

Here's the input we used for a test run on Nov 11, 2026:

{
  "channels": ["durov", "telegram"],
  "maxPostsPerChannel": 100,
  "includeMedia": true
}

That run returned 200 posts from 2 of 2 channels in about a minute.

Step 3: Look at the output

Each post is one dataset item. Here's a real item from that run, trimmed (media CDN URLs shortened):

{
  "channel": "durov",
  "channelTitle": "Pavel Durov",
  "channelSubscribers": 10000000,
  "postId": 554,
  "url": "https://t.me/durov/554",
  "datetime": "2026-11-10T12:59:06+00:00",
  "text": "Any Telegram message can now become an app. ...",
  "views": 993000,
  "isForwarded": false,
  "isReply": false,
  "links": ["https://t.me/BotNews/122"],
  "hashtags": [],
  "media": [{ "type": "video", "url": "https://cdn4.telesco.pe/file/...mp4", "thumbnailUrl": "https://cdn4.telesco.pe/file/..." }],
  "reactions": [{ "emoji": "⭐", "isPaidStars": true, "count": 4870 }, "..."]
}

Notes on the data:

views are abbreviated on Telegram's side ("993K" becomes 993000), so treat them as rounded. Very new posts keep gaining views.
channelSubscribers is also Telegram's rounded public figure.
Media URLs are temporary Telegram CDN links. If you need the files, download them soon after the run.

Step 4: Export to CSV or Excel

In the run's Output tab, pick a view and click Export. You can choose CSV, Excel (XLSX), JSON, XML or HTML. The views are:

View	Columns
Overview	channel, postId, datetime, text, views, mediaTypes, isForwarded, url
Engagement	channel, channelSubscribers, postId, datetime, views, reactions, url
Links & media	channel, postId, links, linkPreview, media, forwardedFrom, replyTo, url

Overview is the best fit for a spreadsheet. Nested fields like reactions and media are flattened into columns in CSV. Use JSON if you want to keep them as lists.

Step 5: Automate it from Python or the command line

To skip the browser, call the Actor through the Apify API. Get your API token from Apify Console → Settings → API & Integrations.

curl: this endpoint runs the scraper and returns the results as CSV in one call (for runs up to 5 minutes):

curl -X POST \
  "https://api.apify.com/v2/acts/signalwharf~telegram-channel-scraper/run-sync-get-dataset-items?format=csv&view=overview" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"channels": ["durov"], "maxPostsPerChannel": 50}' \
  -o durov.csv

Python (pip install apify-client):

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("signalwharf/telegram-channel-scraper").call(
    run_input={"channels": ["durov", "telegram"], "maxPostsPerChannel": 100}
)

# Save as CSV
csv_bytes = client.dataset(run["defaultDatasetId"]).get_items_as_bytes(item_format="csv")
open("telegram_posts.csv", "wb").write(csv_bytes)

# Or work with the items directly
for post in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(post["datetime"], post["views"], post["url"])

Step 6: Monitor a channel for new posts

If you want only the posts published since the last run, turn on Only new posts (onlyNewPosts: true). The scraper remembers the last post it saw for each channel. The state lives in a named key-value store in your own Apify account. Then create a Schedule in Apify Console, for example every 15 minutes or every morning, and connect it to Google Sheets, Slack, a webhook, Make or Zapier using Apify's integrations.

Example monitoring input with a keyword filter:

{
  "channels": ["durov", "telegram"],
  "onlyNewPosts": true,
  "keywords": ["update", "launch"],
  "maxPostsPerChannel": 50
}

A monitoring run that finds nothing new charges nothing.

How much does it cost?

The Actor uses pay-per-event pricing, with no monthly fee:

Event	Price
Channel with results (once per channel per run, only if it yields at least one post)	$0.002
Post saved	$0.0015 ($1.50 per 1,000 posts)

Our 200-post test run cost 2 × $0.002 + 200 × $0.0015 = $0.304. One channel with 100 posts costs about $0.15. Filtered-out posts, empty channels and failed channels aren't charged. You can set a maximum cost per run in the run options, and the scraper stops cleanly when it reaches it. Apify's free plan includes a monthly platform credit, which covers small tests like these. Check Apify's pricing page for the current amount.

Troubleshooting

Channel returns preview_disabled: the owner turned off the web preview. There's no way around this without the official API.
Fewer posts than requested: the channel has fewer posts, or your date or keyword filters removed them.
views is null: some service messages have no view count.
Very deep history is slow: each page is about 20 posts, fetched with a polite delay (1.5 s by default). Thousands of posts take a few minutes.

Is it legal to scrape Telegram?

This approach reads only what Telegram publishes openly at t.me/s/. It doesn't log in, bypass access controls, or touch private messages or member lists. Posts can still contain personal data, so make sure your use complies with the GDPR and other privacy laws, Telegram's Terms of Service, and copyright. When in doubt, ask a lawyer.

Related tools

Substack Scraper: newsletter posts, comments and subscriber estimates.
AI Brand Visibility Tracker: see how ChatGPT, Claude and Gemini talk about your brand.

Built and maintained by Signal Wharf. Telegram is a trademark of its owner; this tool is not affiliated with or endorsed by Telegram.