DEV Community: veenkes

How to Export Substack Newsletter Posts

veenkes — Fri, 13 Nov 2026 06:03:14 +0000

How to Export Substack Newsletter Posts

Substack is now a primary publishing platform, not just a mailing list. If you research a niche, track competitors, or want an archive of a newsletter you already read, it is reasonable to want a structured export: titles, dates, free versus paid, and engagement. Substack does not offer a simple 'download every post as CSV' button for any publication. This guide shows how to export Substack newsletter posts, including publications that run on a custom domain.

You will use the Substack Scraper on Apify. It collects post metadata (title, URL, date, audience, authors, tags, likes, comments, restacks, word count) and can optionally add the plain text of free posts. Paywalled content is never accessed.

By the end of this article you will have a CSV or JSON file on your machine, and a short Python snippet you can rerun or schedule. The same Actor is also callable from JavaScript and from AI agents through the Apify MCP server.

What you need

An Apify account. The Free plan is enough to try a small run.
A list of inputs for the Actor (URLs, IDs or search terms — details in the steps below).
Optional: Python 3 with the apify-client package if you want to automate the run.

No Substack login is required. You only need the public URL of the publication or of a single post.

How to run it in the Apify Console

Step 1: Open the Actor

Open https://apify.com/veenkes/veenkes-substack-posts and click Try for free. A free Apify account is enough to run a first export.

Step 2: Paste the publication URL

Use the newsletter's home page, whether it lives on *.substack.com or a custom domain such as https://www.lennysnewsletter.com. You can also paste individual post URLs.

Step 3: Choose how many posts, and whether you need full text

Set a max posts value for a first run (10 is plenty). Leave full text off if you only need an inventory of titles and engagement; turn it on only for free posts when you need the body.

Step 4: Export

Start the run. When it finishes, download the dataset as CSV or JSON. Filter later by audience (free vs paid) or by date.

The since date and the audience filter are useful for incremental archives. Full text is billed as a separate event, only for free posts you actually retrieve.

How to run it from Python

Install the official client and call the Actor by its Store id veenkes/veenkes-substack-posts. Replace YOUR_APIFY_TOKEN with a token from Apify Console > Integrations. Do not commit the token to a public repository.

pip install apify-client

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("veenkes/veenkes-substack-posts").call(run_input={
    "publications": [
        "https://www.lennysnewsletter.com"
    ],
    "maxPostsPerPublication": 50,
    "includeContent": False
})

# Write every dataset item, then export from the Console or keep going in code.
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

The JavaScript client (apify-client on npm) uses the same Actor id and the same input object. You can also start the run with a plain HTTP POST to the Apify API if you prefer.

Sample output

Here is a shortened example of one dataset record (field values are illustrative of the schema, taken from a real sample run shape):

{
  "publicationUrl": "https://www.lennysnewsletter.com",
  "postId": 224206488,
  "title": "State of the tech job market: late 2026",
  "subtitle": "PMs are winning, engineers are plateauing, designers are hurting, and the biggest AI hiring boom isn’t in software",
  "url": "https://www.lennysnewsletter.com/p/state-of-the-tech-job-market-late",
  "publishedAt": "2026-11-10T15:04:03.497Z",
  "type": "newsletter",
  "audience": "paid",
  "authors": [
    {
      "name": "Lenny Rachitsky",
      "handle": "lenny"
    }
  ],
  "tags": [
    "Career"
  ],
  "section": null,
  "wordCount": 2183,
  "likes": 323,
  "comments": 33,
  "restacks": 30,
  "description": "PMs are winning, engineers are plateauing, designers are hurting, and the biggest AI hiring boom isn’t in software",
  "previewText": "👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. For more: Lenny’s Jobs | Lenny’s Podcast | Lennybot | How I AI | Become an AI-Nativ…",
  "contentText": null,
  "coverImage": "https://substackcdn.com/image/fetch/$s_!-6sL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8be609d0-ed10-464c-8b94…",
  "podcastUrl": null,
  "scrapedAt": "2026-11-12T20:04:50.732Z"
}

Once the run has finished, open the dataset in the Console and use Export > CSV if a spreadsheet is your destination. JSON is usually better for code and for language models.

What to use the data for

Writers use this to audit their own archive or to study how often a newsletter hits the paywall. Researchers track topic frequency and engagement over time. Growth teams watch a set of newsletters in a niche and note which posts get restacks and comments. Because the schema is flat, you can load the file into a spreadsheet or a notebook and join it with your own annotations.

Pricing

Pay-per-result pricing starts at $1.50 per 1,000 posts on the Apify Free plan. Adding the full text of a free post is a separate event (from $1.00 per 1,000). There is also a small per-run start fee. Paywalled bodies are never accessed. Always check the live pricing on the Actor page, because plans and unit prices can change. For a first test, keep the input tiny so you can inspect the schema before you scale up.

Scheduling and automations

In the Apify Console you can schedule the Actor (daily or hourly) and leave the same input in place. For incremental jobs, prefer a "since" or "posted after" style filter when the Actor offers one, so you are not paying to re-collect old rows. When a run finishes you can send a webhook, or connect the Actor in Zapier, Make or n8n. AI agents can discover and call veenkes/veenkes-substack-posts through Apify's MCP server using the same input.

Good practices and limits

Only publicly visible metadata (and optional free-post text) is collected. Do not treat this as a way to bypass a paywall. Respect the writer's copyright: an export for research, analysis or personal archiving is a different thing from republishing someone else's issues. The Actor is not affiliated with Substack.

If a run returns fewer rows than you expected, check the input caps, language or country filters, and the Actor log. A zero-row run is usually a filter or an unavailable source, not a silent crash — read the log before you increase limits.

Next step

Open the Actor and run a small job: Substack Scraper: Export Newsletter Posts.

If you later want this on a schedule, save the input that worked, add a date filter where it applies, and export CSV automatically from the dataset or from the API snippet above.

How to Scrape Shopify Products to CSV

veenkes — Fri, 13 Nov 2026 03:07:02 +0000

How to Scrape Shopify Products to CSV

Every public Shopify store exposes a product catalog that is good enough to shop from — and, if you are careful and respectful, good enough to research. Price monitoring, assortment analysis, affiliate catalogs and competitor scoring all start from the same question: can I get this store's products, variants, prices and stock into a CSV? This guide shows how, without a Shopify admin login.

You will use the Shopify Product Scraper on Apify. Point it at a store or a collection URL and export titles, vendors, types, tags, prices, compare-at prices, discount percent, availability, variants, SKUs and images. You can choose one row per product or one row per variant (size, colour, SKU).

What you need

An Apify account. The Free plan is enough to try a small run.
A list of inputs for the Actor (URLs, IDs or search terms — details in the steps below).
Optional: Python 3 with the apify-client package if you want to automate the run.

The Actor reads the store's public catalog (including the well-known products.json feed when the merchant leaves it available). You do not need the merchant's password or an admin API token.

How to run it in the Apify Console

Step 1: Open the Actor

Open https://apify.com/veenkes/veenkes-shopify-products and click Try for free.

Step 2: Paste store URLs

A homepage such as https://colourpop.com works. A /collections/... URL limits the run to that collection.

Step 3: Choose row shape and a market

Use product rows for a catalog overview, or variant rows for price and stock monitoring. Set the country if you need market-specific currency and pricing.

Step 4: Run and export CSV

Cap max products on the first run. When the dataset is ready, export CSV or JSON. Turn on descriptions only if you need the body copy.

Only available skips sold-out items. Updated since is the right filter for incremental monitoring so you are not re-exporting an unchanged catalog every day.

How to run it from Python

Install the official client and call the Actor by its Store id veenkes/veenkes-shopify-products. Replace YOUR_APIFY_TOKEN with a token from Apify Console > Integrations. Do not commit the token to a public repository.

pip install apify-client

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("veenkes/veenkes-shopify-products").call(run_input={
    "stores": [
        "https://colourpop.com"
    ],
    "maxProductsPerStore": 100,
    "outputLevel": "product"
})

# Write every dataset item, then export from the Console or keep going in code.
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

The JavaScript client (apify-client on npm) uses the same Actor id and the same input object. You can also start the run with a plain HTTP POST to the Apify API if you prefer.

Sample output

Here is a shortened example of one dataset record (field values are illustrative of the schema, taken from a real sample run shape):

{
  "store": "https://www.tentree.com",
  "storeName": "Tentree",
  "collection": null,
  "productId": 8143544320186,
  "title": "Lookout Long Puffer",
  "handle": "lookout-long-puffer-forest-pine",
  "url": "https://www.tentree.com/products/lookout-long-puffer-forest-pine",
  "vendor": "tentree",
  "productType": "Womens",
  "tags": [
    "back-to-school",
    "best-sellers",
    "bogo-tote"
  ],
  "currency": "USD",
  "market": "US",
  "createdAt": "2025-06-06T10:20:29-07:00",
  "publishedAt": "2026-11-11T14:23:07-07:00",
  "updatedAt": "2026-11-12T14:01:32-07:00",
  "imageUrl": "https://cdn.shopify.com/s/files/1/2341/3995/files/Green-Cloud-Shell-Long-Puffer-TCW3532-5590_11.jpg?v=1749230441",
  "price": 348,
  "priceMax": 348,
  "compareAtPrice": null,
  "discountPercent": null,
  "available": true,
  "variantsCount": 5,
  "options": [
    {
      "name": "Color",
      "values": [
        "FOREST PINE"
      ]
    },
    {
      "name": "Size",
      "values": [
        "XS",
        "S",
        "M"
      ]
    }
  ],
  "variants": [
    {
      "id": 44905051095226,
      "title": "FOREST PINE / XS",
      "sku": "TCW3532-5590-XS",
      "price": 348,
      "compareAtPrice": null,
      "available": false
    },
    {
      "id": 44905051127994,
      "title": "FOREST PINE / S",
      "sku": "TCW3532-5590-S",
      "price": 348,
      "compareAtPrice": null,
      "available": false
    },
    {
      "id": 44905051160762,
      "title": "FOREST PINE / M",
      "sku": "TCW3532-5590-M",
      "price": 348,
      "compareAtPrice": null,
      "available": false
    }
  …
}

Once the run has finished, open the dataset in the Console and use Export > CSV if a spreadsheet is your destination. JSON is usually better for code and for language models.

What to use the data for

Brands track competitors' prices and discounting. Retail analysts map an assortment by vendor and product type. Affiliate and comparison sites keep a fresh product file with images and canonical URLs. Because variant mode is one row per SKU, restock and price-change jobs are straightforward to schedule.

Pricing

Pay-per-result pricing starts at $1.00 per 1,000 products on the Apify Free plan, with lower unit prices on paid Apify plans, plus a small per-run start fee. Always check the live pricing on the Actor page, because plans and unit prices can change. For a first test, keep the input tiny so you can inspect the schema before you scale up.

Scheduling and automations

In the Apify Console you can schedule the Actor (daily or hourly) and leave the same input in place. For incremental jobs, prefer a "since" or "posted after" style filter when the Actor offers one, so you are not paying to re-collect old rows. When a run finishes you can send a webhook, or connect the Actor in Zapier, Make or n8n. AI agents can discover and call veenkes/veenkes-shopify-products through Apify's MCP server using the same input.

Good practices and limits

Only scrape public storefront data, and respect the merchant's terms and robots rules. Do not use this to copy a store's creative assets or descriptions into a competing shop. Some merchants disable public JSON endpoints; results then depend on what the storefront still exposes. The Actor is not affiliated with Shopify.

Next step

Open the Actor and run a small job: Shopify Product Scraper: Prices, Variants & Stock.

If you later want this on a schedule, save the input that worked, add a date filter where it applies, and export CSV automatically from the dataset or from the API snippet above.

How to Scrape Jobs from Greenhouse, Lever, Ashby and Workable

veenkes — Fri, 13 Nov 2026 00:53:52 +0000

How to Scrape Jobs from Greenhouse, Lever, Ashby and Workable

A large share of tech and growth-stage hiring still runs through a handful of applicant tracking systems: Greenhouse, Lever, Ashby, Workable and SmartRecruiters. Each company gets a public board URL, and those boards are the most reliable place to see current openings — better than a stale aggregate job site. If you want those jobs in one spreadsheet, you should not have to write a parser per ATS. This guide shows how to scrape them in a single run.

You will use the ATS Job Scraper on Apify. Paste career-board URLs; the Actor detects the ATS automatically and returns a normalized schema: title, location, remote flag, department, salary (when the board publishes it), dates and the full description.

What you need

An Apify account. The Free plan is enough to try a small run.
A list of inputs for the Actor (URLs, IDs or search terms — details in the steps below).
Optional: Python 3 with the apify-client package if you want to automate the run.

You do not need an ATS API key or a login. Public board URLs are enough.

How to run it in the Apify Console

Step 1: Open the Actor

Go to https://apify.com/veenkes/veenkes-ats-jobs and click Try for free.

Step 2: Paste board URLs

Examples: https://boards.greenhouse.io/stripe or https://jobs.ashbyhq.com/ramp. Mix Greenhouse, Lever, Ashby, Workable and SmartRecruiters in the same run if you want.

Step 3: Optional filters

Add keywords, a location string, remote-only, or a posted since date if you are building a daily new-jobs alert. Set a modest max jobs per company for the first run.

Step 4: Run and export

Start the run and export the dataset as CSV or JSON. Full descriptions are included as plain text and HTML at no extra charge.

If a URL is not a supported board, the run will skip or report it rather than silently inventing jobs. You pay only for jobs that are actually saved.

How to run it from Python

Install the official client and call the Actor by its Store id veenkes/veenkes-ats-jobs. Replace YOUR_APIFY_TOKEN with a token from Apify Console > Integrations. Do not commit the token to a public repository.

pip install apify-client

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("veenkes/veenkes-ats-jobs").call(run_input={
    "companies": [
        "https://boards.greenhouse.io/stripe"
    ],
    "maxJobsPerCompany": 50,
    "includeDescription": True
})

# Write every dataset item, then export from the Console or keep going in code.
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

The JavaScript client (apify-client on npm) uses the same Actor id and the same input object. You can also start the run with a plain HTTP POST to the Apify API if you prefer.

Sample output

Here is a shortened example of one dataset record (field values are illustrative of the schema, taken from a real sample run shape):

{
  "platform": "ashby",
  "company": "ramp",
  "companyName": null,
  "jobId": "b7fd55f4-6c5a-42f5-be04-aa09a4d5b7a9",
  "title": "Senior Design Engineer",
  "url": "https://jobs.ashbyhq.com/ramp/b7fd55f4-6c5a-42f5-be04-aa09a4d5b7a9",
  "applyUrl": "https://jobs.ashbyhq.com/ramp/b7fd55f4-6c5a-42f5-be04-aa09a4d5b7a9/application",
  "location": "Remote (US)",
  "allLocations": [
    "Remote (US)",
    "San Francisco, CA",
    "New York, NY (HQ)"
  ],
  "country": "United States",
  "remote": true,
  "workplaceType": "Remote",
  "employmentType": "FullTime",
  "department": "Engineering",
  "team": "Engineering",
  "salaryText": "$188,000 - $259,250",
  "salaryMin": 188000,
  "salaryMax": 259250,
  "salaryCurrency": "USD",
  "salaryInterval": "1 YEAR",
  "publishedAt": "2026-11-12T20:01:54.072Z",
  "updatedAt": null,
  "descriptionText": "ABOUT RAMP\n\nRamp is building the smart infrastructure for finance teams, embedded in the transaction flow of every dollar a business spends. We automate how over $275B in annualize…",
  "descriptionHtml": "<h1><strong>About Ramp</strong></h1><p style=\"min-height:1.5em\">Ramp is building the smart infrastructure for finance teams, embedded in the transaction flow of every dollar a busi…",
  "scrapedAt": "2026-11-12T20:38:24.077Z"
}

Once the run has finished, open the dataset in the Console and use Export > CSV if a spreadsheet is your destination. JSON is usually better for code and for language models.

What to use the data for

Recruiters watch a list of target companies and get new roles the day they go live. Founders and compensation researchers compare titles, locations and published salaries. Lead-generation teams treat a burst of hiring in a department as a buying signal. Job-board products use a normalized schema so Greenhouse and Ashby jobs can live in the same table.

Pricing

Pay-per-result pricing starts at $2.00 per 1,000 saved jobs on the Apify Free plan, with lower unit prices on paid Apify plans. Full descriptions are included. There is also a small per-run start fee. Always check the live pricing on the Actor page, because plans and unit prices can change. For a first test, keep the input tiny so you can inspect the schema before you scale up.

Scheduling and automations

In the Apify Console you can schedule the Actor (daily or hourly) and leave the same input in place. For incremental jobs, prefer a "since" or "posted after" style filter when the Actor offers one, so you are not paying to re-collect old rows. When a run finishes you can send a webhook, or connect the Actor in Zapier, Make or n8n. AI agents can discover and call veenkes/veenkes-ats-jobs through Apify's MCP server using the same input.

Good practices and limits

Job descriptions are written by the employer and may be copyrighted; use them for search, analysis or application workflows, not to republish a competing board wholesale. Always follow the company's and the ATS vendor's terms. The Actor is not affiliated with Greenhouse, Lever, Ashby, Workable or SmartRecruiters.

Next step

Open the Actor and run a small job: ATS Job Scraper: Greenhouse, Lever, Ashby & Workable.

If you later want this on a schedule, save the input that worked, add a date filter where it applies, and export CSV automatically from the dataset or from the API snippet above.

How to Scrape Google Play Reviews to CSV

veenkes — Fri, 13 Nov 2026 00:53:10 +0000

How to Scrape Google Play Reviews to CSV

If you ship an Android app, Play Store reviews are one of the few places users tell you — in their own words — why they stay, churn, or leave one star. The official Play Console is fine for your own app, but it will not give you a clean CSV of a competitor's reviews, and it is awkward to pipe into a spreadsheet or a sentiment model. This guide shows how to scrape Google Play reviews to CSV (or JSON) in a few minutes, then how to do the same from Python.

You will use the Google Play Reviews Scraper on Apify. It accepts app IDs or Play Store URLs, can filter by star rating, language, country and date, and writes one flat row per review: rating, text, date, thumbs-up count, app version and the developer's reply.

What you need

An Apify account. The Free plan is enough to try a small run.
A list of inputs for the Actor (URLs, IDs or search terms — details in the steps below).
Optional: Python 3 with the apify-client package if you want to automate the run.

You do not need a Google account, a browser automation script, or a Play Developer API key. The Actor reads the public review pages of the Play Store.

How to run it in the Apify Console

Step 1: Open the Actor

Go to https://apify.com/veenkes/veenkes-google-play-reviews and click Try for free. If you do not have an Apify account, create one; the Free plan includes monthly usage credit you can spend on this run.

Step 2: Add the apps

In the input form, paste a package name such as com.spotify.music or a full Play Store URL. You can add several apps in one run.

Step 3: Set a review cap and filters

Set Maximum reviews per app to something small first — 50 or 100 is enough to check the output. Optionally filter to 1–2 star reviews, pick a language (en, de, es, …), a store country, or a since date if you only want new reviews.

Step 4: Run and export CSV

Click Start, wait for the run to finish, then open the dataset and choose Export > CSV. You can also export JSON, Excel or XML. Each row is one review, ready for a spreadsheet or a pandas DataFrame.

Useful filters: Only reviews with text drops star-only reviews; Reviews since plus Newest sort is a good daily incremental job; reviewer names are off by default if you want to stay conservative on personal data.

How to run it from Python

Install the official client and call the Actor by its Store id veenkes/veenkes-google-play-reviews. Replace YOUR_APIFY_TOKEN with a token from Apify Console > Integrations. Do not commit the token to a public repository.

pip install apify-client

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("veenkes/veenkes-google-play-reviews").call(run_input={
    "apps": [
        "com.spotify.music"
    ],
    "maxReviewsPerApp": 100,
    "language": "en"
})

# Write every dataset item, then export from the Console or keep going in code.
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

The JavaScript client (apify-client on npm) uses the same Actor id and the same input object. You can also start the run with a plain HTTP POST to the Apify API if you prefer.

Sample output

Here is a shortened example of one dataset record (field values are illustrative of the schema, taken from a real sample run shape):

{
  "appId": "com.spotify.music",
  "appTitle": "Spotify: Music and Podcasts",
  "appDeveloper": "Spotify AB",
  "appScore": 4.3483925,
  "appRatingsCount": 36631501,
  "appReviewsCount": 1858962,
  "appInstalls": "1,000,000,000+",
  "appGenre": "Music & Audio",
  "appVersion": "VARY",
  "reviewId": "13b5c818-6009-49e7-af2e-2a076f33aaf5",
  "reviewUrl": "https://play.google.com/store/apps/details?id=com.spotify.music&reviewId=13b5c818-6009-49e7-af2e-2a076f33aaf5",
  "rating": 2,
  "title": null,
  "text": "id rather this than apple but the quality of the app has gone down so much since ai, its almost as if features introduced were written by ai and not tested. random features break f…",
  "date": "2026-11-11T19:54:47.254Z",
  "thumbsUp": 0,
  "appVersionReviewed": "9.1.98.621",
  "developerReply": null,
  "developerReplyDate": null,
  "language": "en",
  "country": "us",
  "scrapedAt": "2026-11-12T20:02:33.121Z"
}

Once the run has finished, open the dataset in the Console and use Export > CSV if a spreadsheet is your destination. JSON is usually better for code and for language models.

What to use the data for

Typical uses include ASO (what phrases show up in 1-star reviews), product research (which versions triggered a wave of complaints), competitor monitoring, and building a training set for a sentiment or topic model. Because the rows are flat, you can schedule a daily run, keep only reviews newer than yesterday, and append the CSV to a warehouse.

Pricing

Pay-per-result pricing starts at $0.25 per 1,000 reviews on the Apify Free plan, with lower unit prices on paid Apify plans, plus a small per-run start fee. Always check the live pricing on the Actor page, because plans and unit prices can change. For a first test, keep the input tiny so you can inspect the schema before you scale up.

Scheduling and automations

In the Apify Console you can schedule the Actor (daily or hourly) and leave the same input in place. For incremental jobs, prefer a "since" or "posted after" style filter when the Actor offers one, so you are not paying to re-collect old rows. When a run finishes you can send a webhook, or connect the Actor in Zapier, Make or n8n. AI agents can discover and call veenkes/veenkes-google-play-reviews through Apify's MCP server using the same input.

Good practices and limits

Reviews are public, but they may still contain personal data if you enable reviewer info, and the text itself can be copyrighted by the authors. Use the data in line with Google Play's terms and applicable law. This is not an official Google API, and the Actor is not affiliated with Google.

Next step

Open the Actor and run a small job: Google Play Reviews Scraper.

If you later want this on a schedule, save the input that worked, add a date filter where it applies, and export CSV automatically from the dataset or from the API snippet above.