DEV Community: knot crochet

Don't Put Self-Hosted Config in Environment Variables

knot crochet — Thu, 17 Sep 2026 20:01:27 +0000

Look at the .env.example of any self-hosted app you've installed. Mine used to look like that too: S3 keys, SMTP credentials, payment provider secrets, an ecommerce API token, all of it required, all of it read once at boot.

That design has a specific victim, and it isn't you. It's the person who installs your software but doesn't have a shell on the box it runs on. Every configuration change becomes: edit a file over SSH, restart the container, hope nothing else was depending on the old value. For software that ships an admin UI, that's an incoherent story. Half the settings are in a web form and the other half require a terminal.

I moved essentially all of it into the database. Environment variables are now a fallback, not the source of truth.

The shape

export async function getConfig<T = unknown>(key: string, envFallback?: T): Promise<T | undefined> {
  const mk = memoKey(key);
  const cached = _configMemo.get(mk);
  if (cached && Date.now() < cached.expiresAt) return (cached.value as T) ?? envFallback;

  const { getEffectiveConfig } = makePlatform();
  const value = await getEffectiveConfig.get(key);
  _configMemo.set(mk, { value, expiresAt: Date.now() + CONFIG_MEMO_TTL_MS });
  return (value as T) ?? envFallback;
}

Read the stored value; fall back to the environment variable if it's unset. Keys are declared once in a registry with their env fallback beside them:

DEFAULT_CDN_URL:            k<string>('cdn.default_url',            'DEFAULT_CDN_URL'),
GOOGLE_MAPS_API_KEY:        k<string>('google_maps.api_key',        'GOOGLE_MAPS_API_KEY'),
PERMISSIONS_ADMIN_USER_IDS: k<string>('permissions.admin_user_ids', 'PERMISSIONS_ADMIN_USER_IDS'),

The registry is the part I'd copy into any project. One file that lists every configurable thing, its storage key, its env fallback and its type. It doubles as the documentation, and unlike documentation it can't drift, because the accessors are generated from it.

Four things this got right, and one it didn't

Nothing is required to boot. Every integration is inert until configured. No LLM key means the AI features hide themselves; no S3 config means uploads throw a specific, catchable error and the admin UI shows a banner. The app starts with an empty environment, which is what makes a one-command install possible at all. That property is downstream of this decision, not of the Dockerfile.

Changing a credential doesn't need a restart. For a single-box deployment, that's the difference between a settings form and an SSH session.

A 5-second memo, and it's deliberately that short. Every typed accessor calls getConfig, so a single request can hit it dozens of times, each doing two serial reads. The memo collapses that to one. The TTL is far shorter than any request, so it's effectively per-request, and cross-process staleness is bounded at five seconds. Writes invalidate the exact key in-process, so same-process reads are read-your-writes. I'd rather bound staleness with a small number than build an invalidation protocol across processes.

Nested updates are read-modify-write in application code:

// Read-modify-write of a nested path inside a JSON config document. Done in JS (not
// jsonb_set) because the underlying store does not create missing intermediate containers.
export async function setConfigPath(key: string, path: string[], value: unknown): Promise<void> {
  const doc = (await getConfig<Record<string, unknown>>(key)) ?? {};
  let cursor = doc;
  for (let i = 0; i < path.length - 1; i++) {
    if (cursor[path[i]] === null || typeof cursor[path[i]] !== 'object') cursor[path[i]] = {};
    cursor = cursor[path[i]] as Record<string, unknown>;
  }
  cursor[path[path.length - 1]] = value;
  await setConfig(key, doc);
}

Here's the one it got wrong, or at least the sharp edge I'd warn you about. getConfig returns the memoized object by reference. setConfigPath mutates that object in place before writing it back. So any decision you make based on a config object read before a path-mutating call may be reading post-mutation state. It has bitten me. The rule that keeps it safe is: snapshot what you need before calling anything that mutates a path. A defensive clone in getConfig would remove the whole class, at the cost of an allocation per call, and if I were writing it today I'd pay that.

Deleting a key writes null rather than deleting the row:

// The platform config store is write-through with no row delete; clearing a key to null
// reads back as "unset" (getConfig falls through to envFallback), which is the behavior
// every caller of deleteConfig relies on.

Worth stating because it's the kind of implementation detail that becomes a bug when someone later "optimizes" it into a real delete and changes what unset means.

The one rule I'd write in capitals

// Once published, KV keys MUST NOT be renamed - installations may have written
// rows against them. Add new keys, never repurpose old ones.

Config keys are a public interface the moment you ship a version. Rename email.from_address to email.sender and every existing installation silently reverts to defaults on upgrade. Nothing errors. Emails go out from the wrong address until someone notices.

Environment variables have the same property, but you get away with more, because a missing env var usually fails loudly at boot. A missing database row falls back to a default and keeps running. Moving config to a database makes renames strictly more dangerous, and it's the one thing I'd tell someone before they copy this design.

What genuinely belongs in the environment

Not everything moved, and the split isn't arbitrary. Three things stayed:

DATABASE_URL, because you can't read the config table without it.
The session secret and the credentials encryption key, because they're needed to decrypt what's in the config table.
The port and the admin hostname, needed before request handling exists.

Everything else is data. The test I use: if the app can start without it, it isn't an environment variable. That leaves you with three or four, which is a .env.example someone will actually read.

I Built a Page Importer That Clones Any URL Into an Editor

knot crochet — Tue, 15 Sep 2026 20:01:02 +0000

"Import from URL" is a button in my page editor. You paste any public page's address, and a few seconds later an editable copy of it is open in the canvas, with every image, stylesheet and script already served from your own CDN.

The feature description is one sentence. The implementation is a fetch, a bot-wall detector, an HTML parser, a bounded upload pool, and a network egress policy. Here's what each of those is defending against.

Two tiers, and only one of them fails loudly

Plain fetch gets you the HTML the server sent. It does not get you anything a framework rendered client-side, which on modern marketing pages is often the hero, the tabs and the countdown. So there's a second tier using a headless browser API that returns the DOM after load.

The interesting part is the fallback policy, because "fall back on error" is wrong here:

async function captureHtml(url: string): Promise<{ html: string; tier: ImportTier }> {
  try {
    const html = await fetchRenderedHtml(url);
    return { html, tier: 'browser-rendering' };
  } catch (err) {
    if (err instanceof BrowserRenderingNotConfiguredError) {
      logger.info('Browser Rendering not configured, falling back to fetch', { url });
      return { html: await plainFetchHtml(url), tier: 'fetch' };
    }
    if (err instanceof BrowserRenderingHttpError) throw err;
    if (err instanceof UnsafeUrlError) throw err;

    logger.warn('Browser Rendering network failure, falling back to fetch', { url, error: err });
    return { html: await plainFetchHtml(url), tier: 'fetch' };
  }
}

Three different treatments for three different causes:

Not configured is a normal state, not an error. Most self-hosted installs never set up the browser API. Degrade silently, log at info.
Network failure is transient. Degrade, log at warn.
An HTTP error from the browser service propagates as a 502. This is the one that matters: if the upstream said 500 or 429, silently falling back means the user gets a visibly worse import and no explanation of why. They'd file a bug about missing images, and the real cause would be a quota.

The result carries the tier that produced it, so the UI can say which path ran. Any automatic degradation the user can't see is a support ticket with the wrong title.

Detecting a bot wall before you save it

Fetch a page behind a challenge and you get 200 OK with a body containing a spinner and a JavaScript puzzle. Store that and you've created a "page" whose content is someone else's captcha, which will confuse the user far more than an error would.

So there's a detector, and it's deliberately a pile of string checks:

const TITLE_MARKERS = ['<title>Just a moment...</title>', '<title>Just a moment…</title>'];
const CLASS_MARKERS = ['class="cf-browser-verification"', 'class="cf-injected-html"'];
const TEXT_MARKERS  = ['Checking your browser before accessing'];
const META_REFRESH_RE = /<meta\s+http-equiv=["']refresh["'][^>]*__cf_chl_tk/i;

const CF_WRAPPER_RE = /id=["']cf-wrapper["']/;
const CF_WRAPPER_MAX_BYTES = 50 * 1024;

Two things I'd point out. The title list contains both an ASCII ellipsis and a Unicode one, because the real page has shipped both and matching one of them is a detector that works until it doesn't.

And the last rule is size-gated: an element id alone is too weak a signal, since a real page could legitimately use it, so it only counts as a challenge if the whole document is under 50 KB. A challenge page is tiny; a real page that happens to contain that id is not. When a heuristic is individually too weak, pairing it with a size or structure constraint is usually cheaper than finding a stronger one.

Rewriting the assets without a serial upload loop

A captured page references 50 to 150 assets. Each needs downloading and re-uploading to the user's own storage, then the HTML needs rewriting to point at the new URLs.

The naive version does that one at a time inside the traversal, and blows the request timeout. The structure that works is three phases:

// Collect every absolute source URL first, then upload uniques with a bounded pool, then rewrite
// synchronously off the resolved map.
const pending = new Set<string>();

Collect (a Set, so a logo referenced twelve times uploads once), upload with a concurrency limit of 5, then rewrite synchronously from the resolved map. The rewrite phase touching no network at all is what makes it comprehensible: after the pool drains, it's pure string substitution over a parsed tree.

The attribute list is longer than you'd guess, and each entry is a bug someone would otherwise report:

{ tag: 'img', attr: 'srcset' }, { tag: 'link[rel="apple-touch-icon"]', attr: 'href' },
{ tag: 'video', attr: 'poster' }, { tag: 'source', attr: 'srcset' },
{ tag: 'object', attr: 'data' }, { tag: 'embed', attr: 'src' },

plus a regex pass for url(...) inside inline styles. srcset needs its own parsing branch, since it's a comma-separated list of URL-plus-descriptor pairs rather than a single URL.

The security problem this feature is, by definition

A form that takes a URL and makes the server fetch it is server-side request forgery with a nice UI. Left unguarded, someone types http://169.254.169.254/... or an internal hostname and my server dutifully retrieves it.

So every outbound fetch in this path goes through a wrapper that validates the destination and then, critically, pins the connection to the address it validated. Checking DNS and then calling fetch is not sufficient: the name can resolve differently on the second lookup, which is the DNS rebinding attack. Every redirect hop is re-validated for the same reason.

The runtime split is worth stating plainly, because it's the honest version:

On Node, the connection is pinned to the validated IP while preserving SNI and the Host header, so certificate validation is unaffected.
On workerd there is no node:http and its fetch can't be told which address to use. The request proceeds with a debug log, relying on the platform's egress not reaching a private network.
Any other runtime, with neither DNS nor pinning available, is refused outright.

That last one is the part I'd argue for hardest. The tempting default is to allow when you can't check. A guard that silently disables itself on unknown infrastructure is worse than no guard, because it's the one you stop thinking about. Fail closed and let the deployment tell you it needs support.

There are also byte and time ceilings on the fetch itself (20 MB, 30 seconds), enforced by capping the response body stream rather than trusting the Content-Length header, since a hostile server will happily lie about that.

Two things I tried that made it worse

Scrolling the page before capture. It loads lazy images. It also looks exactly like automation and reliably triggered challenges on sites that had let a plain load through. Net negative.

Inlining computed styles. This gives you a pixel-accurate snapshot and destroys the page, because computed styles are resolved at one viewport width. Every media query stops mattering and the import is unusable on mobile. Snapshot fidelity and editability are in direct conflict, and for something that opens in an editor, editability has to win.

Stop Hardcoding Payment Providers Into Your Checkout Analytics

knot crochet — Thu, 10 Sep 2026 20:01:14 +0000

Here is a code smell that took me two payment providers to see clearly. My funnel analytics page had a column called stripePayment. It also had paypalPayment, paypalCcPayment, and stripeDeclined. The SQL that filled those columns named the event strings inline. The order detail page had a switch that turned a payment method into a badge color. The settings form that collects API keys had a hand-written <input> per provider per field.

Adding a third provider meant touching five files that have nothing to do with payments. Worse, it meant a new contributor had to know those five files existed. That's not a refactor you postpone. That's a design error you fix before it compounds.

The fix is not interesting in the abstract - it's a registry. What's interesting is what belongs in the registration, because my first attempt put too little in it.

What a provider actually owns

export interface PaymentProviderRegistration {
  paymentMethod: string;
  dbProvider: string;
  eventTypes: PaymentEventTypes;
  display: PaymentDisplayInfo;
  displayName: string;
  formFields: PaymentFormFieldSpec[];
  details: Record<string, PaymentDetailInfo>;
  statsColumns: StatsColumnDef[];
  getPublicConfig?: (credentials: Record<string, any>, settings: Record<string, any>) => Record<string, any> | null;
}

The first version of this had eventTypes and nothing else, and it didn't help much, because the analytics page still had to know which columns to draw. The version that actually removed provider names from unrelated modules is the one where the provider also declares its own analytics columns:

statsColumns: statColumns([
  ['click',   'PAYPAL_BUTTON_CLICK',  'paypalButtonClick',  'PayPal',    'PayPal Button Click'],
  ['click',   'PAYPAL_EXPRESS_CLICK', 'paypalExpressClick', 'PP Express','PayPal Express Button Click'],
  ['click',   'PAYPAL_CC_CLICK',      'paypalCcClick',      'PP CC',     'PayPal Credit Card Click'],
  ['success', 'PAYPAL_PAYMENT',       'paypalPayment',      'PayPal',    'PayPal Payment Success'],
  ['success', 'PAYPAL_CC_PAYMENT',    'paypalCcPayment',    'PP CC',     'PayPal Credit Card Payment Success'],
  ['error',   'PAYPAL_ERROR',         'paypalError',        'PP Error',  'PayPal Payment Error'],
]),

and the analytics page asks the registry rather than a hardcoded list:

export function getAllStatsColumns(): StatsColumnDef[] {
  return flatMapEntries((reg) => reg.statsColumns);
}

Now "add a provider" is one object in one file. The stats table grows a column. The settings screen grows a form. The order badge renders. Nothing in src/lib/stats contains the string "Stripe".

The detail that makes this non-obvious: a provider is not a payment method

This is the part I'd have gotten wrong if I'd designed it up front instead of after shipping.

PayPal is not one thing. A buyer can pay through the PayPal button, through PayPal Express, or by typing a card number into PayPal's embedded card fields. Commercially those are three different things: if your embedded card fields are broken, your PayPal revenue looks fine in aggregate and you lose money for a week without noticing.

So PAYPAL_PAYMENT and PAYPAL_CC_PAYMENT are separate success events, chosen at write time from metadata:

export function getSuccessEventType(paymentMethod: string, metadata?: Record<string, any>): string {
  const provider = getProviderByMethod(paymentMethod);
  if (!provider) return FALLBACK_SUCCESS;
  const { successEvents } = provider.eventTypes;
  const wantsCardEvent = paymentMethod === 'paypal' && Boolean(metadata?.isCardPayment);
  if (wantsCardEvent && successEvents.length > 1) return successEvents[1];
  return successEvents[0] ?? FALLBACK_SUCCESS;
}

I'm not going to pretend that's clean. paymentMethod === 'paypal' is a provider name inside the registry's generic resolver, which is exactly what the registry was supposed to eliminate. It's the honest state of the code: one provider has a sub-method distinction that the others don't, and I chose a three-line special case over a general "sub-method selector" abstraction that would have one implementation.

If a second provider ever needs it, the special case becomes a selectSuccessEvent(metadata) hook on the registration and this function loses the if. Until then, generalizing would be inventing a requirement.

Fall back, don't throw

Every one of these lookups takes a string that came from a client request or a database row written months ago by an older version of the code:

const FALLBACK_SUCCESS = 'PAYMENT_SUCCESS';
const FALLBACK_ERROR = 'PAYMENT_ERROR';

export function getPaymentDisplay(paymentMethod: string): PaymentDisplayInfo {
  const provider = getProviderByMethod(paymentMethod);
  if (provider) return provider.display;
  return { ...FALLBACK_DISPLAY, label: paymentMethod };
}

An unknown method renders its own raw name in a grey badge. It does not throw. This matters more than it looks: the alternative is that deleting a provider from the registry makes every historical order that used it un-viewable, and you find that out in production, from a customer, during a refund.

Registration is a side effect, and that has a cost

for (const registration of [paypal, stripe]) {
  registerPaymentProvider(registration);
}

A module-level loop mutating a module-level Map. It's the simplest thing that works, and it has a real failure mode: if some module reads getAllStatsColumns() before this file has been imported, it gets an empty array and the stats page silently renders zero columns. Import order becomes load-bearing.

I'd flag this as the weakest part of the design. The fix, if it ever bites, is an explicit initPaymentProviders() called from the composition root instead of an import side effect. I haven't done it because there's exactly one entry point today, and I'd rather have the known-shaped problem than a speculative fix.

When this is over-engineering

If you will only ever integrate one payment provider, skip all of it. The break-even isn't the second provider - it's the second provider whose event shape differs from the first. Two providers that both emit one click and one success event can live behind a plain enum forever.

I hit the break-even the moment PayPal needed three click events and Stripe needed one, because that's the point where every consumer of the data has to stop assuming a one-to-one mapping. That assumption was baked into four files. Registering it in one was cheaper than deleting it from four again later.

I Built an Astro Integration That Injects Its Own Pages Directory

knot crochet — Tue, 08 Sep 2026 20:00:42 +0000

Most Astro integrations add a renderer, inject one endpoint, or wire a Vite plugin. Mine injects an entire application: every admin page, every storefront route, every API endpoint, all shipped inside an npm package so a consumer project can mount them with one line and then override any of them.

integrations: [autonnel({ excludeRoutes: ['/login'], plugins: [oauth2()] })]

The whole integration is 261 lines across eight files. Here's how it works and, more usefully, where the approach leaks.

Enumerating your own pages at config time

injectRoute takes a pattern and an entrypoint, one call at a time. I have 203 routable files and I'm not writing that list by hand, so the integration walks its own src/pages directory during astro:config:setup and derives the patterns the same way Astro's file router would:

function derivePattern(relPath: string): string {
  let p = relPath.replace(/\\/g, '/').replace(/\.(astro|ts|js)$/, '');
  p = p.replace(/\/index$/, '');
  if (p === 'index') p = '';
  return '/' + p;
}

overview.astro becomes /overview. orders/index.astro becomes /orders. api/permissions/roles/[id].ts becomes /api/permissions/roles/[id], dynamic segment intact, because the bracket syntax is already the pattern syntax. That's the part that makes this viable: file-router conventions and route-pattern syntax are the same language, so the translation is three string operations.

The walk has to reproduce the router's exclusions too:

if (name.startsWith('_') || name.startsWith('.')) continue;   // Astro ignores these
if (!/\.(astro|ts|js)$/.test(name)) continue;
if (/\.test\.(ts|js)$/.test(name)) continue;
if (/\.d\.ts$/.test(name)) continue;
if (name === '404.astro') continue;                            // consumer owns its own 404

That last one is a product decision hiding in a loop. Injecting a 404 page means the consuming app can never have its own, and a 404 is exactly the sort of page a consumer wants to brand. Excluding one file is cheaper than an option nobody would find.

The dual-path detail: the same code has to work from source during development and from dist/ after publish, so the pages directory is resolved from where the integration file itself lives.

const pagesRel = integrationUrl.includes('/dist/') ? '../../src/pages/' : '../pages/';

A string check against your own module URL is not elegant. It is, as far as I can tell, the shortest thing that actually works for a package that must behave identically before and after a build step.

Exclusion is the whole point

export function filterRoutes(routes: InjectableRoute[], exclude: string[]): InjectableRoute[] {
  if (!exclude || exclude.length === 0) return routes;
  const set = new Set(exclude);
  return routes.filter((r) => !set.has(r.pattern));
}

Eight lines, and they're the reason this design works at all. A consumer that wants a different login page passes excludeRoutes: ['/login'] and puts their own src/pages/login.astro in their project. The core route is never injected, so there's no conflict to resolve and no precedence rule to remember.

Plugin routes deliberately bypass the filter:

// These bypass excludeRoutes: a consumer typically excludes a core route (e.g. /login) so a
// plugin can serve its own page at that pattern.
for (const plugin of resolved.plugins) {
  for (const route of plugin.routes ?? []) injectRoute({ pattern: route.pattern, entrypoint: route.entrypoint });
}

The common case is exclude-then-replace, so making the two lists interact would break the only workflow anyone uses.

The part I'd flag in review

The comment I'm least happy with is in index.ts:

The injected pages are core SOURCE files importing via @/...; the consumer must resolve that alias to the package src and mark the package src ssr.noExternal. Astro's updateConfig does not reliably apply a resolver plugin / alias from here, so this stays consumer-side.

That is an integration with a documentation dependency. Install it, and it doesn't work until you also add an alias and an ssr.noExternal entry to your own config. I tried to push both into updateConfig and couldn't make it apply reliably from the setup hook.

If you're designing something similar, know that this is the sharp edge: injectRoute will happily take an entrypoint that resolves inside your package, but the imports inside that entrypoint resolve in the consumer's module graph, not yours. Any non-relative import in an injected page is a constraint you're imposing on the consumer, and there's no mechanism to declare it.

The alternative is compiling injected pages to use relative or bare specifiers only. That's a real fix and it's a large refactor of a hundred files, so I've written down the constraint instead of paying for it. That's a trade, not a solution, and I'd want a reviewer to call it that.

The gap I found writing this post

The integration is also where plugins get registered:

registerAuthoringComponents();
registerPlugins(resolved.plugins);

Which means plugins only register when the app is consumed through the integration. My two documented install paths (a Docker image and a git clone) both run the repo's own astro.config.mjs and use the native file router, not the integration. So on both of them, registerPlugins never runs and a marketplace plugin can't activate.

That's a real gap, not a hypothetical one, and I found it by writing out the flow for this post rather than by reading the code. There's a lesson in that which I'd generalize: any initialization that lives in an optional wrapper is unreachable on every path that doesn't use the wrapper. Registration belongs in the composition root that all paths share. Mine doesn't, yet.

Would I recommend this shape

For "reusable pieces", no. Ship components and let people compose them.

For "an entire application that a consumer extends", the injected-routes approach is genuinely good, and the exclusion mechanism is what makes it feel like a framework rather than a fork. The honest caveats: your injected pages' imports become a contract you can't enforce, and any setup work that happens in the integration hook silently doesn't happen for anyone running your source directly.

I Made a Self-Hosted Checkout Stack That Boots With One Command and Zero Config

knot crochet — Thu, 03 Sep 2026 20:00:41 +0000

Most self-hosted software fails its user in the first ten minutes, and it fails in a specific way: .env.example has forty variables, eight of them are required, and three of those require you to go create an account somewhere else before the app will boot.

I set a hard constraint for mine: docker compose up, open a browser, done. No Node install, no external Postgres, no S3 bucket, no SMTP credentials, no generated secrets. Not "fast to configure" - zero configuration to boot.

That constraint turned out to be an architecture decision, not a packaging decision, and it changed how config works in the app itself.

The compose file is three services

services:
  db:
    image: postgres:17-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U autonnel -d autonnel"]
      interval: 3s
      retries: 20

  schema:
    image: ghcr.io/example/app:${TAG:-latest}
    depends_on:
      db:
        condition: service_healthy
    command:
      - node_modules/.bin/prisma
      - db
      - push
      - --schema=./prisma/schema.prisma
      - --url
      - *db-url
    restart: "no"

  app:
    image: ghcr.io/example/app:${TAG:-latest}
    depends_on:
      schema:
        condition: service_completed_successfully
    ports: ["${PORT:-4321}:4321"]

The middle one is the piece people skip. schema is a one-shot container that creates or updates tables and exits. app waits on service_completed_successfully, not on service_started.

Two properties fall out of that, and both matter more than the ten lines cost:

The app never races the schema. There is no retry loop, no "waiting for database" spinner, no partially-migrated first request.
The upgrade path is the same command as the install path. docker compose up after a version bump runs the migration container again before starting the new app. There is no separate migrate step to document, and therefore no step for a user to skip.

The restart: "no" is not decoration. Without it, a one-shot container that exits 0 gets restarted by a default restart policy inherited from anywhere, and you get a migration loop.

The gotcha that only appears in a stripped image

Prisma 7 reads the datasource URL from prisma.config.ts. That file is a build-time artifact and it is not in my runtime image, because the runtime image only contains dist, node_modules, package.json and the schema file.

So the migration command has to pass the URL explicitly:

command: [..., --url, "postgresql://user:pass@db:5432/app"]

That took me longer to diagnose than it should have, because the error is about a missing config file and the cause is an image layer decision made in a different file. It's worth stating the general shape: the smaller you make your runtime image, the more implicit build-time context you have to make explicit at run time. Anything your tooling "just finds" in the repo needs to be passed as a flag once the repo isn't there.

For the same reason, prisma generate runs in the build stages with a dummy URL fallback in the config. Generation never connects to a database, but it will refuse to run without a syntactically valid URL, and there is no database at image-build time.

Zero config and real secrets are not actually in conflict

The uncomfortable part of a zero-config boot is session secrets and credential encryption keys. You can't ask for them and still boot in one command. You can't generate them per-boot or every restart logs everyone out and makes stored provider credentials unreadable.

So: baked development defaults, overridable, with the warning at the point of use rather than in a doc.

AUTH_SESSION_SECRET: ${AUTH_SESSION_SECRET:-dev-only-insecure-session-secret-change-me}
CREDENTIALS_ENCRYPTION_KEY: ${CREDENTIALS_ENCRYPTION_KEY:-ZGV2LW9ubHktaW5zZWN1cmUta2V5LWNoYW5nZS1tZQ==}

Compose reads a neighbouring .env automatically, and values there win, so "make this production-ready" is: create .env, put two openssl rand outputs in it. That instruction lives in a comment at the top of the compose file, next to the sentence that actually gets read:

Rotating either value invalidates existing sessions and makes stored provider credentials unreadable, so generate once and keep them stable.

The default value is literally the string dev-only-insecure-.... If it ever shows up in a support thread or a screenshot, it identifies itself.

Why nothing is required to boot

The zero-config property isn't really a packaging trick. It's downstream of one decision made inside the app: S3 credentials, the email provider, payment keys and the ecommerce backend are not environment variables. They're rows in a config table, edited in the admin UI, with env as a fallback. Every integration is therefore inert until configured, and the features that need one degrade instead of crashing.

That decision has its own trade-offs, enough of them that they don't fit here. For this post the only relevant consequence is the one above: an app whose integrations are all optional at boot is an app that can boot with an empty environment.

The image itself

Three stages, standard shape, one detail worth copying:

deps: production dependencies only, plus prisma generate.
builder: full dependencies, prisma generate again, then the app build.
runner: copies node_modules from deps and dist from builder.

Copying the production module tree from a stage that never installed dev dependencies is what keeps the final image from carrying a build toolchain. It costs one extra install during build and it's worth it.

Plus a healthcheck that hits a real endpoint, USER node, and HOST=0.0.0.0 (without which the server binds to localhost inside the container and the port mapping appears to do nothing - a five-minute mystery everyone gets to solve once).

What I'm not claiming

This is a single Postgres container with a named volume. It is a genuinely good way to run a small deployment on one box, and it is not a highly-available production database. If you're at the point of needing that, point DATABASE_URL at your managed instance and delete the db service - the compose file is designed so that's a two-line edit rather than a fork.

How I Got One Astro Codebase to Run on Node and Cloudflare Workers

knot crochet — Tue, 01 Sep 2026 20:02:16 +0000

I ship the same application to two runtimes: Node, for people who self-host with Docker, and Cloudflare Workers, for people who want it on the edge. Same repo, same source files, one astro.config.mjs and one astro.config.cloudflare.mjs.

That worked. It cost me about six distinct debugging sessions to get there, and every one of them was a case where the second config had to say something the first one didn't. Here they are, because none of them are in a getting-started guide.

The two configs are 90% identical, and that's the problem

The obvious approach is one config with an if. I started there and moved away, because the divergences aren't a single flag - they're a different adapter, a different externals policy, a different alias set, and a different set of build memory knobs. An if ladder covering all of that is harder to read than two files.

The cost is real and I want to name it: two files that must stay in sync in the parts where they agree. The place this bites is React resolution, and it bites hard enough that the Cloudflare config carries a comment saying so:

// Must mirror astro.config.mjs's React handling. Without dedupe the
// production Rollup client build resolves react-dom's internal react to a
// different chunk than the islands' react, yielding two React instances ->
// "Cannot read properties of null (reading 'useEffect')" when IslandHydrator
// calls createRoot().render() on a hooked component.

If you take one thing from this post: resolve.dedupe for React is not an optimization. Two React copies in one bundle fail at the first hook call, with an error message that points at your component and not at your bundler.

Prisma's generated client doesn't resolve for workerd

Prisma 7 generates a client that uses Node.js subpath imports (#main-entry-point). Rollup, targeting workerd, can't resolve that. The fix is aliasing straight to the edge entry:

const PRISMA_CLIENT_DIR = path.dirname(require.resolve('@prisma/client/package.json'));
const PRISMA_EDGE_ENTRY = path.resolve(PRISMA_CLIENT_DIR, '../../.prisma/client/edge.js');

resolve: {
  alias: {
    '.prisma/client/default': PRISMA_EDGE_ENTRY,
  },
}

Note what that path resolution is doing. In Prisma 7.8+ the generated .prisma/client/ lives inside the @prisma/client package directory, and with pnpm that's .pnpm/@prisma+client@<hash>/node_modules/.prisma/client/, not the project's top-level node_modules. Hardcoding a path works on your machine and breaks on any machine with a different hoisting layout or store hash. Resolving relative to @prisma/client/package.json is the version that survives.

The externals list, and one entry that is deliberately absent

Node-only packages must not go into the workerd bundle. Mine is short:

const NODE_ONLY_EXTERNALS = ['ioredis'];

ioredis is loaded via dynamic import() behind an isCloudflareRuntime() guard, so its chunk is never fetched on Cloudflare and externalizing it is safe.

pg is not in that list, and the comment explaining why is the single most useful comment in the file:

@prisma/adapter-pg imports pg statically and PrismaPg runs on the CF Hyperdrive path, so pg must be bundled. With nodejs_compat enabled, pg's TCP connection works through Cloudflare's Node.js compatibility layer. Externalizing it produced Uncaught Error: No such module "chunks/pg" at worker load time.

The general rule I extracted: a package is externalizable only if every path that imports it is dynamic and guarded. One static import anywhere in the graph and externalizing turns a build-time question into a load-time crash, which is much worse because it happens after deploy.

The adapter overwrites your externals, so re-append them

This one took the longest to find. @astrojs/cloudflare sets vite.ssr.noExternal = true and vite.build.rollupOptions.external = ['sharp'] inside its own astro:build:setup hook. Anything you put in ssr.external is simply gone by the time Rollup runs.

The workaround is a post-enforced plugin that re-appends:

{
  name: 'autonnel:cf-extra-externals',
  enforce: 'post',
  config(conf) {
    const existing = conf.build?.rollupOptions?.external;
    if (Array.isArray(existing)) {
      conf.build.rollupOptions.external = [...new Set([...existing, ...NODE_ONLY_EXTERNALS])];
    } else if (typeof existing === 'function') {
      const existingFn = existing;
      conf.build.rollupOptions.external = (id, parentId, isResolved) =>
        NODE_ONLY_EXTERNALS.includes(id) || existingFn(id, parentId, isResolved);
    }
    // ...string / RegExp / undefined branches
  },
}

Four branches, because external can legitimately be an array, a string, a RegExp, a function, or absent, and I don't control what the adapter will set next release. The comment above it says "remove once @astrojs/cloudflare stops clobbering ssr.external", which is the honest status: this is a workaround against a specific version's behavior, and it's written to be deletable.

The build ran out of memory in CI and not locally

Bundling every SSR entrypoint into one workerd bundle peaked above Node's 2 GB default heap. It OOM'd at about 1.99 GB on Cloudflare's build CI while succeeding on my machine, which is the least fun failure mode there is.

Two lines fixed it:

build: {
  sourcemap: false,
  reportCompressedSize: false,
},

Sourcemaps were the biggest single consumer, and workerd doesn't consume them anyway. reportCompressedSize allocates a gzipped copy of every chunk purely to print a nicer table at the end of the build. Both are pure cost in this target.

The worker entry does two things Node's doesn't have to

The Node build gets a request lifecycle for free. On Workers I own it:

export default {
  async fetch(request, env, ctx) {
    setRuntimeEnv(env);
    return runWithRequestDb(async () => {
      try {
        return await ssrHandler.fetch(request, env, ctx);
      } finally {
        ctx.waitUntil(disposeRequestDb());
      }
    });
  },
  async scheduled(_event, env) { /* ... */ },
};

setRuntimeEnv(env) exists because on Workers there is no process.env. Bindings arrive as an argument to the handler, which means anything that reads configuration has to go through an indirection that is populated per-invocation. If you're porting a Node app, this is the change that touches the most files, and doing it early is much cheaper than doing it after you've written a hundred process.env.FOOs.

ctx.waitUntil(disposeRequestDb()) is the other half: the connection has to be released after the response is returned, not before, or you'll dispose a client that streaming code still needs.

Would I do it again

Yes, but I'd be clearer about what the second target is for. Running on Workers is not a free "also deploy anywhere" checkbox. It's a second build with its own failure modes, and roughly all of them surface at deploy time rather than in tests.

What makes it worth it is that the divergence is confined: two config files and one entry file. No if (isWorkers) scattered through business logic, because everything runtime-specific is behind an adapter (cache, storage, database) that already existed for other reasons. If your app doesn't have those seams yet, build them first and add the second runtime after. Doing it in the other order is how you end up with runtime checks in a checkout service.

I Rebuilt ClickFunnels' One-Click Upsell. The Hard Part Wasn't the UI.

knot crochet — Thu, 27 Aug 2026 20:01:37 +0000

One-click upsell is the page that appears right after someone buys: "add this for $29, no need to re-enter your card." ClickFunnels built a business on it. I rebuilt it, and I want to be precise about where the work actually was, because the visible part is a button and a price.

The button took an afternoon. The four failure orderings behind it took weeks.

Problem one: the two providers have opposite shapes

I support Stripe and PayPal. "Charge the customer again without asking for the card" means two genuinely different things.

Stripe. Vault the payment method during the base checkout, then create a second, independent charge later. The vaulting happens at intent creation:

// Vault the card for later one-click upsell charges: a Customer + setup_future_usage=off_session
// makes the confirmed payment method reusable off-session.
setup_future_usage: customerId ? 'off_session' : undefined,

with one rule that is not negotiable: vaulting is best-effort and must never block the base sale.

} catch {
  customerId = undefined; // vaulting is best-effort; never block the base sale
}

If creating the Customer object fails, the buyer still buys. They just don't get offered upsells. Losing the $79 base order to protect a hypothetical $29 upsell is the wrong trade, and it's an easy one to get backwards when you're writing the upsell feature and the upsell feels like the point.

PayPal. There is no second charge. The order sits approved-but-not-captured, and an upsell is a PATCH that raises the amount, followed by exactly one capture when the buyer leaves the upsell chain:

// PayPal: the order is approved-but-not-captured. PATCH it to add the upsell amount + record the
// line in the snapshot (no charge). When the buyer leaves the upsell chain, capture once - the order
// then materializes with base + all accepted upsells.

So for one provider "accept upsell" means money moved. For the other it means a number in a pending authorization changed and money hasn't moved at all, including for the base order.

I tried to hide this behind one chargeUpsell() port. Don't. The abstraction has to sit further out, at the HTTP boundary, where both paths converge on the same response shape (success, action, order, nextStepUrl). Everything below that is two flows, and pretending otherwise produces a port whose contract is a lie for one of its implementations.

Problem two: idempotency, twice, differently

A buyer double-clicks. A phone loses signal mid-request and the browser retries. This is not an edge case on a page whose entire design encourages one impulsive click.

Stripe gets an idempotency key scoped to the specific upsell step:

idempotencyKey: `${tenantId}:${saleRef}:upsell:${upsellIndex}`

and the local ledger row is written with the same key, with the unique-violation swallowed:

} catch (err) {
  if ((err as { code?: string })?.code !== 'P2002') throw err;
}

PayPal can't use an idempotency key for this, because the operation isn't "create a charge", it's "set the total to a new value". A retried PATCH with the same intent would be indistinguishable from a legitimate second upsell at the same price. So idempotency is a state check against the snapshot:

if (!snapshot.lines.some((l) => l.upsellIndex === upsellIndex)) {
  // append the line, recompute the total, patch the order
}

Same guarantee, completely different mechanism, because the underlying operation is set rather than add.

Problem three: getting it into the merchant's store

This is the part nobody writes about and the part that generates support tickets.

The merchant's actual fulfillment happens in Shopify or WooCommerce. So a sale with three accepted upsells has to arrive there. As what, exactly?

Two legitimate answers, and I made it a setting rather than picking one:

merged: base plus all accepted upsells as one external order. One shipment, one packing slip. This is what a single-warehouse merchant wants.
split: the base as one order, plus one standalone order per upsell, each tagged back to the parent. This is what a merchant with per-supplier dropshipping needs, because those items ship from different places.

The dispatch is one function, and the interesting lines are the two short-circuits:

export async function executeHandoffForSale(saleRef: string, deps: HandoffForSaleDeps): Promise<void> {
  const mode = await deps.getFulfillmentMode();
  if (mode !== 'split') {
    await deps.runMergedHandoff(saleRef);
    return;
  }
  const ctx = await deps.loadSplitContext(saleRef);
  if (!ctx || ctx.upsellLines.length === 0) {
    await deps.runMergedHandoff(saleRef);
    return;
  }
  await deps.runBaseOnlyHandoff(saleRef);
  for (const u of ctx.upsellLines) {
    try {
      await deps.pushIndependentUpsell({ saleRef, upsellIndex: u.upsellIndex, parentOrderNumber: ctx.orderNumber, line: u.line, customer: ctx.customer });
    } catch (err) {
      logger.error('split upsell push failed', { error: err, saleRef, upsellIndex: u.upsellIndex });
    }
  }
}

A sale with zero upsells is byte-identical in both modes, so it takes the merged path regardless of the setting. That's not an optimization, it's a correctness statement: the mode should not be able to produce two different results for the same input.

And the loop is deliberately best-effort. One failing sub-order must not prevent the other sub-orders or the main order from existing. A retry job picks up the failure; a thrown exception would strand the whole sale.

Problem four: the one that actually loses money

Here's the ordering that took the longest to see.

In merged mode the push to the merchant's store is held until the buyer finishes the upsell chain, so the order can be assembled complete. Fine. But the buyer isn't obligated to follow that script. They can close the tab, come back from an email an hour later, and accept an upsell after the main order has already been pushed.

Now the upsell cannot be folded into an order that already exists downstream. If you do nothing, the charge succeeds and the item is never fulfilled. The customer paid and gets nothing. That is the worst failure mode in the entire system, and it is completely silent - your payment numbers look great.

The handling is a degrade, not an error:

} else {
  // The main order was already pushed to the backend (handoff not deferred), so this late upsell
  // cannot be folded into it - push it as an independent external order tagged to the parent.
  // Covers merged late-degrade and split late-accept; prevents the upsell from being lost.
  await pushUpsellIndependently(saleRef, intent, line, upsellIndex).catch((err) =>
    logger.error('Independent upsell push failed (cron will retry)', { error: err, saleRef, upsellIndex }));
}

A merchant configured for merged fulfillment gets a second order they didn't ask for, tagged parent:<mainOrderNumber> so it's traceable. That's strictly better than the alternative, and the tag is what makes it defensible when they ask why.

The ordering rule I'd keep

Charge, then persist, then push, and treat everything after the charge as retryable:

const order = await appendLineToOrder(saleRef, line, locals).catch((err) => {
  logger.error('Upsell charged but appending to order failed', { error: err, saleRef, upsellIndex });
  return undefined;
});

The buyer sees a success page even if the internal bookkeeping failed, because from their side the transaction is complete - it is. The log line is written so that the reconciliation job and the human reading it both know exactly which sale is inconsistent.

The general rule: the irreversible step goes last among things that can fail independently, and everything after it must be retryable and must not be able to fail the response. Every bug I hit in this feature was a violation of that sentence in one direction or another.

The UI is a button. Everything worth writing down is in what happens after someone taps it twice on a train.

I Built Runtime Islands for Astro Because My Pages Live in a Database

knot crochet — Tue, 25 Aug 2026 20:01:27 +0000

Astro's island model has one assumption baked into it: you know at build time which components need JavaScript. You write client:load on a component in a .astro file, the compiler sees it, and it emits exactly the right script tag.

I don't know that at build time. The pages my app renders are drag-and-drop pages. A user arranges blocks in a visual builder, and the result is a JSON tree in Postgres. At build time, the page doesn't exist yet. At request time, I have a tree of maybe forty component instances, of which perhaps three actually need to run in the browser.

So I built runtime islands: server-render the whole tree to static HTML, mark only the interactive nodes, and let a single client script figure out at runtime what to mount and what to download.

The two options I rejected first

Render the whole page as one React root. This is the obvious move and it's what I did first. It works, and it means a landing page with a hero image, an FAQ accordion and a footer ships the entire component library to the browser so that React can re-render markup that will never change.

Mark every component as interactive. Same outcome with more ceremony.

Both fail for the same reason: the ratio. On a typical landing page in this system, the interactive components are the countdown timer and nothing else. On a checkout page, it's the address form, the card form and the coupon field. The static-to-interactive ratio is somewhere around 10:1, and paying for the 10 to get the 1 is the whole thing island architecture exists to avoid.

The server half

Puck (the builder library) takes a config of components and a data tree, and renders it. I wrap the config before rendering: any component whose name is in the interactive set gets its render replaced by one that emits a marker div plus its own props, serialized.

function islandRenderer(componentName: string, renderComponent: (props: any) => any) {
  return function RenderIsland(props: any) {
    return createElement(
      'div',
      { 'data-island': componentName, 'data-island-id': props.id || '' },
      createElement(renderComponent, props),
      serializedPropsScript(props),
    );
  };
}

function wrapInteractiveComponent(name: string, component: any) {
  if (!INTERACTIVE_COMPONENTS.has(name)) return component;
  return { ...component, render: islandRenderer(name, component.render) };
}

Then the page is just renderToStaticMarkup. Note that it's renderToStaticMarkup, not renderToString: there are no hydration markers in the output, because nothing is going to hydrate in React's sense. More on that below.

The props script is a JSON script tag, which means the one escaping rule you cannot skip:

dangerouslySetInnerHTML: {
  __html: JSON.stringify(props).replace(/</g, '\\u003c'),
}

Any user-authored string in those props can contain </script>. If you forget this line, your builder becomes an XSS delivery mechanism the first time someone types the wrong thing into a headline field.

The client half

The entire client-side runtime is one component, mounted once per page with client:only="react". It queries the DOM, collects the distinct island types actually present, and dynamically imports only those:

async function hydrateIslands() {
  const islands = Array.from(document.querySelectorAll<HTMLElement>('[data-island]'));
  if (islands.length === 0) return;

  const components = await loadComponents(collectIslandTypes(islands));

  for (const island of islands) {
    const type = island.getAttribute('data-island');
    const Component = type ? components[type] : null;
    if (!type || !Component) continue;
    const props = readIslandProps(island, type);
    if (props) mountIsland(island, Component, props, 'en');
  }
}

loadComponents walks a plain map of dynamic imports:

const coreLoaders: Record<string, ComponentLoader> = {
  ShippingAddressForm: () => import('./blocks/ShippingAddressForm').then(m => m.ShippingAddressForm),
  PaymentEntryForm:    () => import('./blocks/PaymentEntryForm').then(m => m.PaymentEntryForm),
  CountdownTimer:      () => import('./blocks/CountdownTimer').then(m => m.CountdownTimer),
  // ...
};

The bundler splits every one of those into its own chunk. A page with a countdown timer downloads the countdown chunk. It does not download the Stripe form.

Three things that bit me

1. There are two lists, and they have to agree.

The server decides what to wrap (INTERACTIVE_COMPONENTS). The client decides what can mount (islandLoaders). They live in separate files because they're consumed by separate bundles, and nothing in the type system connects them.

Add a component to the server list only, and you ship a data-island div that never comes alive. It renders correctly, it looks fine, and the button does nothing. That is a genuinely nasty bug class because the failure is silent and visual inspection passes.

I fixed it the only way that actually holds: a test that renders a page through the server path and asserts the island wrapper plus the serialized props survive the round trip.

it('emits an island wrapper carrying default-merged scalar props', () => {
  const html = renderPuckToHtmlWithIslands(pageWith('CountdownTimer'));
  expect(html).toContain('data-island="CountdownTimer"');

  const props = extractIslandProps(html, 'CountdownTimer');
  expect(props.hours).toBe(2);
  expect(props.theme).toBe('block');
});

2. It's createRoot, not hydrateRoot.

This is the part I'd argue about with someone, so let me state the trade honestly. React hydration wants the client's first render to match the server's markup exactly. My props don't come from a React render on the server; they come from a JSON blob that a user edited in a builder. Insisting on hydration means fighting mismatch warnings forever for a class of component that is about to re-render anyway (a countdown timer's server output is wrong by definition the moment it's serialized).

So the server markup is a paint placeholder, and createRoot replaces it on mount. What I get: no mismatch warnings, and one simple contract (props in a script tag) rather than two coupled ones. What I pay: the interactive subtree is rendered twice, and for anything time-dependent there's a visible correction on mount. For a countdown that's fine and arguably correct. If your interactive components are large and stable, real hydration is the better trade and you should take it.

3. Defaults must be merged before serialization.

The builder stores only the props a user explicitly changed. Everything else lives in the component's defaultProps. Server-side, Puck merges those for you during render, so the HTML looks right. But if you serialize the raw stored props, the client mounts the component with half its props undefined and the island renders differently from the markup it just replaced.

The fix is one call (applyPuckDefaults) before rendering, and the assertion props.hours === 2 in that test above exists specifically to catch its removal.

When not to do this

If your pages are known at build time, use client:load and stop reading. This entire mechanism exists to buy back a capability the compiler gives you for free when the page structure is static.

It pays off exactly when the content tree is authored at runtime by someone who is not you: a CMS, a page builder, a template marketplace. In that case the compiler can't help, and the choice is between shipping everything and building this.

The whole runtime is 246 lines across four files, and two of those files are just lists. That is the actual reason I'd recommend it over a framework: it's small enough to read in one sitting, which matters a lot for something sitting between your server output and your users' browsers.