I Generated 41,000 Static Pages with Astro + AI in a Week — Here's the Architecture

Last month I shipped a programmatic SEO site that generates software comparison pages at scale. The site is called ShowdownHQ and it currently serves tens of thousands of "X vs Y for [audience]" comparison pages across note-taking tools, CRM platforms, project management software, and accounting tools.

This is a technical breakdown of how the pipeline actually works.

The Architecture

The system has four components that run in sequence:

Python (generate topics) → Python + LLM API (generate content) → Astro SSG (build static HTML) → Vercel (deploy)

No database. No server-side rendering at request time. Everything is static HTML built at deploy time.

Component 1: Topic Generation

A Python script (generate_topics.py) reads two CSV files — a list of software tools and a list of buyer audiences — and outputs every valid combination as rows in a new CSV.

import csv
import itertools

tools = [("notion", "Notion"), ("obsidian", "Obsidian"), ("craft", "Craft"), ...]
audiences = ["software engineers", "project managers", "creative writers", ...]

with open("data/software_topics.csv", "w") as f:
    writer = csv.writer(f)
    writer.writerow(["slug", "tool_a", "tool_b", "audience", "title"])
    for (tool_a_slug, tool_a_name), (tool_b_slug, tool_b_name) in itertools.combinations(tools, 2):
        for audience in audiences:
            for audience in audiences:
                slug = f"{tool_a_slug}-vs-{tool_b_slug}-for-{audience.replace(' ', '-')}"
                title = f"{tool_a_name} vs {tool_b_name} for {audience.title()}"
                writer.writerow([slug, tool_a_name, tool_b_name, audience, title])

With 8 tools and 15 audiences, this generates 420 rows in under a second. Scale to multiple categories and you're into the tens of thousands of pages before writing a line of content.

Component 2: Content Generation

generate_content.py reads the CSV and, for each row, sends a structured prompt to an LLM API (I use OpenAI's API with GPT-4o) to generate the comparison content.

The prompt is loaded from a .txt file — never hardcoded — and receives the tool names and audience as variables. The output is written as an MDX file to data/articles/.

from pathlib import Path
from openai import OpenAI
import csv

client = OpenAI()
prompt_template = Path("shared-tools/prompts/comparison-article.txt").read_text()

with open("data/software_topics.csv") as f:
    for row in csv.DictReader(f):
        prompt = prompt_template.format(**row)
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": prompt}]
        )
        slug = row["slug"]
        Path(f"data/articles/{slug}.mdx").write_text(response.choices[0].message.content)

The MDX files include frontmatter (title, description, tools, audience) and structured content sections. Total API cost for the initial batch was around $40.

Component 3: Astro SSG

Astro reads the MDX files and builds one static HTML page per article using a dynamic route.

src/pages/[slug].astro

The [slug].astro file uses getStaticPaths() to enumerate every MDX file in data/articles/ and returns the content as props. Astro compiles everything to static HTML at build time. No JavaScript is shipped to the client unless explicitly added.

---
import { getCollection } from 'astro:content';

export async function getStaticPaths() {
  const articles = await getCollection('articles');
  return articles.map(article => ({
    params: { slug: article.slug },
    props: { article }
  }));
}

const { article } = Astro.props;
const { Content } = await article.render();
---

<Layout title={article.data.title}>
  <Content />
</Layout>

Astro's build output is a flat directory of HTML files — exactly what you want for a static site with thousands of pages. Build times are fast because there's no server-side data fetching.

Component 4: Vercel Deployment

The Astro site deploys to Vercel on every push to main. Vercel detects the Astro framework automatically and runs npm run build. The output directory (dist/) gets served via their CDN.

Free tier on Vercel handles this without issue — even at 40,000+ pages, the build artifact is just HTML files.

The Affiliate Link Architecture

All affiliate links go through a centralized redirect handler at /go/[link] rather than being hardcoded in content.

src/pages/go/[link].js

The handler reads from data/links.json, which maps product names to affiliate URLs. When a product's affiliate URL changes, you update one file instead of hunting through thousands of MDX files.

{
  "notion": "https://notion.so/?via=showdownhq",
  "obsidian": "https://obsidian.md/",
  "asana": "https://asana.com/?ref=showdownhq"
}

This also means you can add UTM parameters, swap networks, or redirect to different landing pages without touching content.

What the Build Actually Produces

The first live deployment covers four software categories:

• notes.showdownhq.com — note-taking tools (Notion, Obsidian, Evernote, Craft, Roam, etc.)
• crm.showdownhq.com — CRM platforms (HubSpot, Salesforce, Pipedrive, Zoho)
• pm.showdownhq.com — project management (Asana, Monday, ClickUp, Linear, Notion)
• accounting.showdownhq.com — accounting tools (QuickBooks, Xero, FreshBooks, Wave)

Total pages across all four: 41,000+. Total hosting cost: $0/month on Vercel's hobby tier.

What I'd Do Differently

Use Astro Content Collections from the start. I initially had the MDX files outside the src/content/ directory, which meant I couldn't use Astro's type-safe collection API. Migrating later added friction.

Generate sitemaps during the build, not manually. At this scale, a manually maintained sitemap is immediately stale. Astro's @astrojs/sitemap integration handles this automatically.

Validate MDX frontmatter before committing. A malformed frontmatter block in one file will fail the entire Astro build. I added a Python validation script (validate_content.py) that checks frontmatter structure before the build runs.

The Non-Technical Part

The pipeline is the easy part. The interesting finding from this project was how much the SaaS industry buries real pricing information. Salesforce's real year-one TCO is $180K–$350K for mid-market — the advertised $25/seat is functionally irrelevant. Mailchimp cut their free tier from 2,000 to 250 contacts in January 2026 and the old advice is still everywhere online. Asana made automations unlimited in October 2025; Monday.com still caps at 250/month.

The programmatic part generates pages. The value comes from the manually researched details that make those pages accurate when the SaaS marketing pages aren't.

The site is at ShowdownHQ. Source pipeline architecture is straightforward enough to replicate if you want to build something similar.

Questions on the Astro setup or the generation pipeline welcome in the comments.