Moving blog content to a private repository

Original post: Moving blog content to a private repository

Series: Part of How this blog was built, documenting every decision that shaped this site.

This site's source code is public. Anyone can see how the layouts work, how the
comment system is wired up, how the scheduled build fires. I want that to stay
open: it's useful to others and it keeps me honest about code quality.

What I don't want public is every unpublished draft sitting in the repository
weeks before it goes live. The content, meaning Markdown files, cover images,
and frontmatter, needed to move somewhere private without changing how the
site builds or deploys.

The constraint

Astro's content collections expect post files at a specific path. The collection
loader in src/content.config.ts points to collections/posts/:

loader: glob({ pattern: ["**/*.md", "!README.md"], base: "./collections/posts" }),

Whatever solution I chose, Astro needed to find Markdown files at that path at
build time. No loader changes, no path rewiring, no special plugins.

Why not a git submodule

The obvious answer is a git submodule: point collections/posts/ at a private
repository and let git handle the rest.

It works locally, but Netlify's build pipeline clones
submodules during its "preparing repo" stage, before the build command runs.
For a private submodule, Netlify needs SSH access to the repository. The way
it authenticates is through a deploy key, and GitHub deploy keys are scoped
to a single repository. The same key cannot be added to two repos.

Since Netlify's deploy key was already registered on the main site repository,
adding it to the content repository returned a "key already in use" error. You
can work around this with machine users or manually managed SSH keys, but it
felt like fighting the tooling rather than working with it.

The build-time clone approach

The simpler solution: skip submodules entirely and clone the private repository
as a build step. The content directory is gitignored in the main repository and
fetched fresh on every build.

The entire change is in netlify.toml:

[build]
  command = "git clone --depth 1 https://${GH_PAT}@github.com/sourcier/sourcier.uk-content.git collections/posts && rm -rf collections/posts/.git && astro build"
  publish = "dist"

Three commands chained together:

1. Clone the private content repository into collections/posts/ using a GitHub personal access token for authentication. --depth 1 skips history, since we only need the latest files.
2. Remove .git/ from the cloned directory. Without this, Netlify's secrets scanner detects the token in collections/posts/.git/config and fails the build. Removing the .git directory strips the credential before the scan runs.
3. Build the Astro site as normal. The content is in place and the collection loader picks it up without any configuration changes.

Diagram fallback for Dev.to. View the canonical article for the full version: https://sourcier.uk/blog/private-content-repo

Setting up the GitHub token

The token needs minimal permissions. A fine-grained personal access token scoped
to the content repository with read-only Contents access is sufficient:

1. Go to GitHub → Settings → Personal access tokens → Fine-grained tokens
2. Create a new token scoped to the content repository only
3. Grant Contents → Read-only permission
4. Add the token as GH_PAT in the Netlify dashboard under Site configuration → Environment variables

The token never appears in the repository. It's injected at build time through
the environment variable.

The .gitignore entry

Since the content is cloned at build time, the directory is gitignored in the
main repository:

# blog content (cloned from private repo at build time)
collections/posts/

Local development

For local development, clone the content repository once into collections/posts/:

git clone [e-mail]:sourcier/sourcier.uk-content.git collections/posts

Because the directory is gitignored, the main repository doesn't track it. You
work inside collections/posts/ as its own git repo: commit content changes
there and push to the content repository's remote. The main site repo never
sees those files.

Running pnpm dev picks up the content from disk exactly as before. Nothing
changes about the local workflow except that content commits go to a different
remote.

VS Code workspace setup

With two git repositories on disk, a VS Code multi-root workspace makes the
local setup feel intentional rather than awkward. The .code-workspace file at
the root of the site repository adds collections/posts as a second folder:

{
  "folders": [
    { "name": "sourcier.uk (site)", "path": "." },
    { "name": "sourcier.uk (content)", "path": "collections/posts" }
  ],
  "settings": {
    "terminal.integrated.cwd": "${workspaceFolder:sourcier.uk (site)}"
  }
}

Opening this file gives you both repositories in a single VS Code window. The
Source Control panel shows two separate entries, one for each repo, so staging,
committing, and pushing content changes is completely independent from site
changes. The file explorer shows both roots side by side.

The settings block is optional, but terminal.integrated.cwd is worth adding.
Without it, VS Code opens terminals relative to whichever file is currently
active, which means a terminal opened while editing a post drops you into
collections/posts/ instead of the project root where pnpm dev and other site
commands live.

Because collections/posts/ is gitignored by the site repo, VS Code won't
accidentally stage content files as part of a site commit. Each repository
manages its own history, and the workspace just surfaces both at once.

Triggering deploys from the content repository

Pushing to the content repository doesn't trigger a Netlify deploy: the main
site repository hasn't changed, so Netlify has nothing to react to.

The fix is a Netlify build hook: a URL you POST to kick off a build regardless
of code changes.

1. In the Netlify dashboard go to Site configuration → Build hooks and create a new hook. Copy the URL.
2. In the content repository, add that URL as a secret named NETLIFY_BUILD_HOOK under Settings → Secrets and variables → Actions.
3. Add a workflow to the content repository:

name: Deploy site

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Trigger Netlify build
        run: curl -X POST -d '{}' "${{ secrets.NETLIFY_BUILD_HOOK }}"

Now every push to main in the content repository fires a full Netlify build,
which clones the latest content and rebuilds the site.

What this gives you

Drafts stay private. Unpublished posts, half-written ideas, and cover images
for future content don't appear in the public repository's commit history.

The site code stays open. Layouts, components, serverless functions, and
configuration remain public and visible.

The build is identical. Astro's content collection loader doesn't know or
care where the files came from. They're at the expected path, and the schema
validates them the same way.

No infrastructure changes. No new services, no CMS, no API layer. It's
two git repositories and a one-line build command.

Gating the preview branch behind a passcode

The build hook workflow targets main, so the public site auto-deploys on
content changes. But the preview branch, the one that includes drafts, is
also a public URL. Anyone who finds or guesses preview--sourcieruk.netlify.app
can read unpublished posts.

The solution is a Netlify Edge Function that intercepts every request on the
preview deploy and serves a passcode form until the visitor authenticates.

import type { Config, Context } from "@netlify/edge-functions";

const COOKIE_NAME = "preview_auth";
const COOKIE_MAX_AGE = 60 * 60 * 24 * 7; // 7 days

async function computeToken(passcode: string): Promise<string> {
  const encoder = new TextEncoder();
  const key = await crypto.subtle.importKey(
    "raw",
    encoder.encode(passcode),
    { name: "HMAC", hash: "SHA-256" },
    false,
    ["sign"],
  );
  const signature = await crypto.subtle.sign(
    "HMAC",
    key,
    encoder.encode("preview_authenticated"),
  );
  return btoa(String.fromCharCode(...new Uint8Array(signature)));
}

export default async function previewAuth(request: Request, context: Context) {
  const passcode = Netlify.env.get("PREVIEW_PASSCODE");

  // No passcode configured — pass through (production unaffected)
  if (!passcode) return context.next();

  const url = new URL(request.url);
  const expectedToken = await computeToken(passcode);
  const cookies = parseCookies(request.headers.get("cookie") ?? "");

  if (cookies[COOKIE_NAME] === expectedToken) return context.next();

  if (request.method === "POST") {
    const body = await request.formData();
    const submitted = body.get("code")?.toString() ?? "";

    if (submitted === passcode) {
      const redirectTo = sanitizeRedirect(
        url.searchParams.get("redirect") ?? "/",
        url.origin,
      );
      const response = new Response(null, {
        status: 302,
        headers: { location: redirectTo },
      });
      response.headers.append(
        "set-cookie",
        `${COOKIE_NAME}=${expectedToken}; HttpOnly; Secure; SameSite=Strict; Max-Age=${COOKIE_MAX_AGE}; Path=/`,
      );
      return response;
    }

    return new Response(renderForm(url, true), { status: 401, ... });
  }

  return new Response(renderForm(url, false), { status: 401, ... });
}

export const config: Config = { path: "/*" };

A few things worth calling out:

The cookie stores an HMAC, not the passcode. The passcode is used as an
HMAC key to sign the string "preview_authenticated". The resulting signature
goes into the cookie. The raw passcode never leaves the server: if the cookie
is stolen it reveals nothing about the code, and it's valid only on this origin.

The cookie is HttpOnly, Secure, and SameSite=Strict. It can't be read
by JavaScript, only travels over HTTPS, and is never sent cross-site.

Production is unaffected by design. The function checks for a
PREVIEW_PASSCODE environment variable and no-ops immediately if it isn't set.
The variable is scoped to branch deploys only in the Netlify dashboard, so the
production site never even evaluates the auth logic.

Changing the passcode invalidates all sessions instantly. The HMAC changes
when the key changes, so existing cookies no longer match. No session store, no
token revocation list.

To wire it up:

1. In the Netlify dashboard: Site configuration → Environment variables → Add variable
   • Key: PREVIEW_PASSCODE
   • Value: a short memorable code, or generate one with openssl rand -base64 12
   • Scope: Branch deploys only
2. Install @netlify/edge-functions as a devDependency, this gives TypeScript the types locally. Netlify's edge runtime provides the module automatically at deploy time, so it only needs to be present for type checking:

   pnpm add -D @netlify/edge-functions

1. Netlify auto-discovers edge functions in netlify/edge-functions/, no extra netlify.toml configuration is needed. The path is declared inside the function itself via export const config: Config = { path: "/*" }.

The full edge function (including the form renderer) is in the
site repository.

Purging content from the existing git history

Moving files to a private repository stops future content from appearing in the
public repo. But every post you committed before the split is still visible in
the history. Anyone can check out an older commit and read all your drafts.

The tool for this is git filter-repo.
It rewrites history by replaying every commit through a filter, in this case,
"keep everything except collections/posts/":

brew install git-filter-repo
git filter-repo --path collections/posts --invert-paths --force

--path identifies the directory to target. --invert-paths turns the filter
around: instead of keeping only that path, it removes it. --force is required
because the repo has a remote, and git-filter-repo refuses to run on repos with
remotes unless you explicitly confirm intent.

After the rewrite, git-filter-repo removes the origin remote as a safety
measure. Add it back and force push:

git remote add origin [e-mail]:sourcier/sourcier.uk-content.git
git push --force origin main

If you have other branches (a preview branch, for example), recreate them
from the rewritten main and force push those too:

git branch -D preview
git checkout -b preview
git push --force origin preview
git checkout main

The force push replaces every ref on GitHub with the rewritten chain. The old
SHAs, and the content blobs they referenced, are no longer accessible via the
public repository.

One caveat: GitHub caches some data, such as pull request diffs and web UI
caches, that may retain stale content for a period after the force push. For a
personal repo with no open PRs this expires naturally, but you can contact
GitHub Support to request an immediate cache purge if needed.

What to watch for

README.md in the content repository. Every GitHub repository gets a
README.md at the root. When the content repository is cloned into
collections/posts/, that README.md lands in the loader's base directory.
Astro's glob loader picks it up and tries to validate it against the post
schema, which fails immediately because it has no title, no pubDate, none
of the required frontmatter fields.

The fix is a negation pattern in the glob loader:

loader: glob({ pattern: ["**/*.md", "!README.md"], base: "./collections/posts" }),

This excludes any file named README.md regardless of where the build runs
from. Without this, the build works fine locally (where the directory contains
only posts) but fails on Netlify every time.

Two repositories to manage. Content and code live in separate repos. When
a code change depends on a content change (a new frontmatter field, for
example), both repos need to be updated and the deploy needs both changes
present. In practice this hasn't been an issue: schema changes are infrequent
and easy to coordinate.

Token expiry. Fine-grained tokens have an expiration date. If the token
expires, builds fail silently with a clone authentication error. Set a calendar
reminder or use a long-lived token if your threat model allows it.

Branch alignment. The build command clones the main branch of the content
repository. If you work on a content branch, you'll need to adjust the clone
command or merge to main before deploying.

Working on something similar?

If you're building a content pipeline, managing private content alongside public
code, or setting up deploy automation, I'm available for consulting.
Get in touch via the contact page and tell me what you're working on.