DEV Community: SuperLede

What YouTube's single-page app did to my Chrome extension

SuperLede — Mon, 05 Oct 2026 08:53:38 +0000

I run a small site that checks whether a YouTube channel is in the Partner Program. A few days ago I shipped a Chrome extension for it. The idea fits in one sentence: put a "Check monetization" button under the channel name, click it, and see the verdict plus the signals behind it.

Adding the button was the easy part. Keeping it on the page, attached to the right channel, was not. Nearly every problem traced back to one thing: YouTube is a single-page app, and it doesn't behave like the ones I've built.

Here's what I ran into.

1. The old page is still there, just hidden

If you look for the channel header with querySelector, you can get the previous channel's header. When you navigate inside YouTube, the page you left isn't removed. It stays in the DOM with a hidden attribute so Back can swap it in again.

So every anchor I mount on starts from the visible page:

const ANCHORS = {
  channel: "ytd-browse:not([hidden]) yt-page-header-renderer yt-flexible-actions-view-model",
  video: "ytd-watch-flexy:not([hidden]) ytd-watch-metadata #owner",
};

The same goes for "is my button still mounted?". Checking isConnected isn't enough, because a node can be perfectly connected inside a hidden page. The question has to be "is it attached to the anchor on the page you can actually see?"

2. The URL is the only thing that tells the truth

The <meta> tags in the head don't update when you navigate inside YouTube. The page you first landed on keeps describing itself while you watch something else.

So the content script never reads page metadata to decide what to check. It takes location.href at the moment of the click, normalizes it, and sends that.

3. yt-navigate-finish isn't enough on its own

YouTube fires yt-navigate-finish after in-app navigation, and that's the obvious hook. But YouTube also re-renders the header without navigating anywhere, and when it does, my button goes with it.

I ended up with two triggers: the event, plus a one-second poll that does two cheap checks. If the URL changed, remount. If the URL is the same but the button isn't on the visible anchor anymore, put it back.

Mounting retries every 250 ms and gives up after 15 seconds. If YouTube ships a redesign and the anchor disappears, the button just doesn't show up. Nothing throws, and the toolbar popup still works. I care about that more than it might sound: an extension that starts throwing errors on every page load after a YouTube redesign gets uninstalled.

4. Results can arrive for a page you've already left

A check takes a few seconds. People click Check and then click the next video. Without care, the first channel's result lands on the second channel's page.

Every request carries an incrementing number, and navigation bumps it. When a result comes back, it's discarded unless its number is still the latest and the URL still matches.

5. A closed shadow root, with all: initial

YouTube's CSS is everywhere and very specific. Without isolation, a button you insert picks up YouTube's font sizes and line heights. Mine lives in a closed shadow root, and the host element gets all: initial so nothing cascades in from the page. Positioning is the only thing set on the host.

6. Network calls go through the service worker

The popup can call fetch itself, but then closing the popup halfway through a check kills the request. The server has already counted it against the daily limit, the result never gets cached, and reopening the popup costs the user another check.

So the popup and the in-page button both message the service worker, and it makes the request. That brought two more things with it. If the popup and the button ask about the same channel at the same moment, only one request goes out. And results are cached for ten minutes, so reopening one doesn't spend another check.

The service worker only accepts messages from its own popup and from top-frame content scripts on youtube.com. Anything else is dropped. Requests go out with credentials: "omit", and a timeout is never retried automatically, because the server may have counted it already.

7. Updates orphan your content scripts

When the extension updates or is reloaded, content scripts already running in open tabs keep running but can no longer talk to the extension. chrome.runtime.id becomes undefined. My scripts check for that first, and if it's gone they remove their button and stop polling. Otherwise you're left with a button that does nothing when clicked.

The opposite case: Chrome doesn't inject content scripts into tabs that were already open when the extension was installed. So when you open the popup on a YouTube tab where the content script isn't answering, it tells you to refresh the tab. Without that line, "the button doesn't appear" would be the first bug report.

Why the check runs on a server

The obvious design is to do it all in the browser. You're already on the page, and the data is right there.

I didn't, for two reasons. One of the strongest public signals is whether ads are actually being served on a channel's recent long videos, and in the user's browser that signal is bent by things the extension can't see: YouTube Premium, ad blockers. A server check gives everyone the same answer for the same channel.

The other reason is that the obvious marker in the page doesn't mean what people assume. Several extensions read one flag from the page source and print "monetized". In a small test it showed up on 5 of 19 channels that weren't in the program. YouTube has run ads on non-partner channels since late 2020 in the US and 2021 elsewhere, so seeing ads, or ad plumbing, proves little by itself.

So the extension is a thin client. The server reads public pages, scores the signals and returns a verdict with each signal marked pass, fail or couldn't read.

Privacy, briefly

Nothing is sent while you browse. A channel or video link goes out only when you click Check, open the popup on a channel or video page, or paste a link, and only after you've said yes once.

Checks from the extension also don't feed the public statistics on the site. That one cost a separate table on the server. The shared cache also drives a weekly dataset and a refresh list, and I didn't want extension usage quietly reshaping either of them.

A pack script that says no

Chrome Web Store review takes long enough that I didn't want to find problems there. The pack script refuses to build if the manifest has keys it doesn't expect, if there's an inline script or inline event handler anywhere, or if anything looks like remote code. For the store build it also strips the development key and checks that the API URL points at production. A rejection costs days, and the script costs seconds.

What I'm watching

Two things: whether the button survives the next few YouTube header changes, and what share of real checks come back "unable to determine". That second number is the failure mode I care about most.

If you've built something that lives inside YouTube's DOM, I'd like to know how you handle remounting. A one-second poll works, but it feels like there should be something better.

The extension is free: Monetization Checker for YouTube. More on how it works on the extension page.

pg_dump to S3: a backup script that won't upload a truncated dump

SuperLede — Thu, 01 Oct 2026 05:33:15 +0000

The script below dumps one PostgreSQL database with pg_dump, reads the archive back to make sure it is complete, and uploads it to S3 or any S3-compatible bucket: Cloudflare R2, Backblaze B2, Wasabi, MinIO. It works the same against RDS, Neon, Railway, Supabase, or your own server. Run it daily and you have backups. Run the second script further down weekly and you know they restore.

#!/usr/bin/env bash
# pg-dump-to-s3.sh: dump one PostgreSQL database, check the archive, upload it
# to S3 or any S3-compatible bucket (R2, B2, Wasabi, MinIO).
# From https://backupdrill.com/guides/pg-dump-to-s3 (MIT licensed).
#
# Needs pg_dump and pg_restore of the same major version as the server, and
# the AWS CLI v2. Configure with environment variables:
#   DATABASE_URL     postgresql://user@host:5432/dbname (password in ~/.pgpass)
#   S3_BUCKET        bucket name
#   S3_PREFIX        folder inside the bucket (default: postgres)
#   AGE_RECIPIENT    optional: age public key; the dump is encrypted before upload
#   AWS_ENDPOINT_URL for anything that is not AWS S3, e.g.
#                    https://<account-id>.r2.cloudflarestorage.com
set -euo pipefail

: "${DATABASE_URL:?DATABASE_URL is not set}"
: "${S3_BUCKET:?S3_BUCKET is not set}"
prefix="${S3_PREFIX:-postgres}"
stamp="$(date -u +%Y-%m-%dT%H-%M-%SZ)"

workdir="$(mktemp -d)"
trap 'rm -rf "$workdir"' EXIT
umask 077
dump="$workdir/$stamp.dump"

# 1. Dump. Custom format is compressed, and pg_restore can restore
#    single tables from it or run in parallel.
pg_dump --format=custom --file="$dump" --dbname="$DATABASE_URL"

# 2. Read the whole archive back before it leaves the machine: --list checks
#    the table of contents, --file=/dev/null decompresses every row. A truncated
#    or corrupt file fails here instead of on the day you need it.
pg_restore --list "$dump" > "$workdir/contents.txt"
pg_restore --file=/dev/null "$dump"
tables="$(grep -c ' TABLE DATA ' "$workdir/contents.txt" || true)"
echo "dump OK: $tables tables with data, $(wc -c < "$dump" | tr -d ' ') bytes"

# 3. Optionally encrypt. Only the holder of the matching private key can
#    read it, so a leaked bucket key does not leak the database.
upload="$dump"
if [ -n "${AGE_RECIPIENT:-}" ]; then
  age --recipient "$AGE_RECIPIENT" --output "$dump.age" "$dump"
  upload="$dump.age"
fi
name="$(basename "$upload")"

# 4. Upload the file, then a checksum next to it.
if command -v sha256sum > /dev/null; then
  sum="$(sha256sum "$upload" | cut -d ' ' -f 1)"
else
  sum="$(shasum -a 256 "$upload" | cut -d ' ' -f 1)"
fi
aws s3 cp "$upload" "s3://$S3_BUCKET/$prefix/$name" --only-show-errors
printf '%s  %s\n' "$sum" "$name" \
  | aws s3 cp - "s3://$S3_BUCKET/$prefix/$name.sha256" --only-show-errors
echo "uploaded s3://$S3_BUCKET/$prefix/$name"

It needs pg_dump and pg_restore of the same major version as your server (check with pg_dump --version and select version()). A newer pg_dump can dump an older server, but its archive then needs a newer pg_restore, and restoring that into the older server version runs into errors covered in the pg_restore guide. Matching versions avoids all of it. It also needs the AWS CLI v2, which also talks to non-AWS buckets through AWS_ENDPOINT_URL. We ran every command here against PostgreSQL 17 and a MinIO bucket before publishing it.

Why the script does not pipe pg_dump into aws s3 cp

The one-liner you see everywhere, pg_dump … | aws s3 cp - s3://…, has a failure mode worth knowing. If pg_dump dies partway (a dropped connection, a statement timeout, a full disk on the server), the pipe simply ends. The AWS CLI sees the end of its input and finishes the upload. We tested it: the pipeline exits with an error, and a truncated file is in the bucket under a normal timestamped name, looking like any other backup until the day you try to restore it.

Writing to a temporary file first costs disk space equal to one compressed dump, and buys two things: nothing is uploaded unless pg_dump succeeded, and pg_restore reads the whole archive back before it leaves the machine. --list alone is not enough for that: it only reads the table of contents at the start of the file, and in our test it passed a dump with its end cut off. --file=/dev/null decompresses every row and failed on the same file. If you have no local disk to spare, stream to a key ending in .partial and rename it only after the pipeline succeeds.

A successful run prints two lines:

$ bash pg-dump-to-s3.sh
dump OK: 4 tables with data, 17730 bytes
uploaded s3://acme-db-backups/postgres/2026-09-29T14-06-30Z.dump

Credentials: keep the password out of the connection string

A password inside DATABASE_URL ends up on pg_dump's command line, where any user on the machine can read it in the process list while the dump runs. Put it in a password file instead and leave it out of the URL:

# a dedicated system user runs the backups
sudo useradd --system --create-home --home-dir /var/lib/pgbackup pgbackup

# /etc/pg-backup.pgpass holds one line: host:port:database:user:password
# read -rs keeps the password off the screen and out of shell history
# IFS= keeps leading/trailing spaces; sed escapes \ and : as .pgpass requires
printf 'Password for backup_user: '; IFS= read -rs PGPASS_VALUE; echo
sudo install -m 600 -o pgbackup /dev/null /etc/pg-backup.pgpass
printf 'db.internal:5432:app:backup_user:%s\n' \
  "$(printf '%s' "$PGPASS_VALUE" | sed 's/[\\:]/\\&/g')" \
  | sudo tee /etc/pg-backup.pgpass > /dev/null
unset PGPASS_VALUE

libpq ignores the file if its permissions are wider than 0600. Point to it with PGPASSFILE, as the environment file below does. The database user needs to be able to read every table you back up; on PostgreSQL 14 and later, granting it the pg_read_all_data role is the simplest way to give a dedicated backup user exactly that. One exception: that role does not bypass row-level security, and pg_dump stops with an error on a table with RLS enabled unless the user owns it or has BYPASSRLS (ALTER ROLE backup_user BYPASSRLS, which needs a superuser or, on managed hosts, the platform's admin role).

For the bucket, give the backup machine a key that can write to the backup prefix and nothing else. A key that can also delete means whoever takes over that server can take your backups with it. A write-only key still lets them overwrite an existing backup with a file of the same name, so also turn on bucket versioning (or S3 Object Lock, where your provider offers it): an overwritten backup then stays recoverable as an older version. Cloudflare R2 has neither; its bucket lock rules do the job differently, by refusing overwrites and deletes under a prefix for a retention period. Deletion of old backups is handled by the lifecycle rule further down, which needs no key at all.

Schedule it: systemd timer, cron, or GitHub Actions

All three read the same settings. On a server, keep them in one file that only the backup user can read:

# /etc/pg-backup.env (chmod 600, owned by pgbackup)
# quote the URL: cron sources this with sh, where an unquoted & (as in
# ?sslmode=verify-full&sslrootcert=...) would end the line
DATABASE_URL="postgresql://backup_user@db.internal:5432/app"
PGPASSFILE=/etc/pg-backup.pgpass
S3_BUCKET=acme-db-backups
S3_PREFIX=postgres/app
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
AWS_DEFAULT_REGION=us-east-1
# R2, B2, Wasabi: uncomment and set the provider's S3 endpoint
# AWS_ENDPOINT_URL=https://ACCOUNT_ID.r2.cloudflarestorage.com

systemd timer (recommended on Linux servers)

# /etc/systemd/system/pg-backup.service
[Unit]
Description=pg_dump to S3
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
User=pgbackup
EnvironmentFile=/etc/pg-backup.env
ExecStart=/usr/local/bin/pg-dump-to-s3.sh

# /etc/systemd/system/pg-backup.timer
[Unit]
Description=Daily pg_dump to S3

[Timer]
OnCalendar=*-*-* 03:15:00 UTC
RandomizedDelaySec=10m
Persistent=true

[Install]
WantedBy=timers.target

sudo install -m 0755 pg-dump-to-s3.sh /usr/local/bin/pg-dump-to-s3.sh
sudo systemctl daemon-reload
sudo systemctl enable --now pg-backup.timer
sudo systemctl start pg-backup.service   # one run now, to test
journalctl -u pg-backup.service -n 20

Persistent=true runs a backup that was missed while the machine was off as soon as it is back. RandomizedDelaySec keeps a fleet of servers from dumping at the same second. Output goes to the journal, and a failed run shows up in systemctl --failed.

cron

# /etc/cron.d/pg-backup: daily at 03:15 server time, as user pgbackup
PATH=/usr/local/bin:/usr/bin:/bin
15 3 * * * pgbackup set -a; . /etc/pg-backup.env; set +a; /usr/local/bin/pg-dump-to-s3.sh >> /var/log/pg-backup/backup.log 2>&1

Two cron details break backup scripts. Cron starts jobs with a minimal PATH that usually does not include /usr/local/bin, where the AWS CLI installs, so set it at the top of the file. And cron does not read your environment file, so the line loads it with set -a. Create /var/log/pg-backup owned by the backup user first.

GitHub Actions

Useful when there is no server to run it on, or the database is a managed service. Commit the script to scripts/pg-dump-to-s3.sh, add the values as repository secrets, and use this workflow:

# .github/workflows/pg-backup.yml
name: pg-backup

on:
  schedule:
    - cron: "15 3 * * *" # daily, 03:15 UTC; runs can start late under load
  workflow_dispatch: {}

permissions:
  contents: read

concurrency:
  group: pg-backup
  cancel-in-progress: false

jobs:
  backup:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      # Use the client that matches your server's major version (17 here).
      # Install it from the PostgreSQL apt repo and put it first on PATH;
      # age is only needed if you set AGE_RECIPIENT.
      - name: Install postgresql-client-17 (PGDG)
        run: |
          sudo install -d /usr/share/postgresql-common/pgdg
          curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc \
            | sudo gpg --dearmor -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.gpg
          echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.gpg] \
          https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" \
            | sudo tee /etc/apt/sources.list.d/pgdg.list
          sudo apt-get update
          sudo apt-get install -y postgresql-client-17 age
          echo "/usr/lib/postgresql/17/bin" >> "$GITHUB_PATH"

      - name: Dump and upload
        run: bash scripts/pg-dump-to-s3.sh
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
          S3_BUCKET: ${{ secrets.S3_BUCKET }}
          S3_PREFIX: postgres/app
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          AWS_DEFAULT_REGION: us-east-1
          AGE_RECIPIENT: ${{ secrets.AGE_RECIPIENT }}
          # R2, B2, Wasabi only. Delete this line for AWS S3.
          AWS_ENDPOINT_URL: ${{ secrets.AWS_ENDPOINT_URL }}

The runner image may come with an older PostgreSQL client, which is why the workflow installs a matching version and puts it first on PATH. On a runner the password can stay in DATABASE_URL: the machine is yours alone and discarded after the job. Scheduled runs can start late when GitHub is busy, and GitHub disables schedules in public repositories after 60 days without activity, so check the Actions tab now and then. The Supabase GitHub Actions guide covers the egress side of running backups from GitHub in more detail.

Encrypt the dump before upload

S3 server-side encryption protects the disks at Amazon. It does nothing against the more likely leak: someone with a read key for the bucket sees your database in plain text. Encrypting on the backup machine with a public key closes that gap, because the machine that writes backups cannot read them:

# once, on your own machine: create a key pair
age-keygen --output pg-backup-key.txt
age-keygen -y pg-backup-key.txt   # prints the public key, age1...

# on the backup server, add the public key to /etc/pg-backup.env
AGE_RECIPIENT=age1...

With AGE_RECIPIENT set, the script uploads …dump.age instead of the plain dump. Keep the private key somewhere that is not the backup server and not the bucket, such as a password manager. Losing it means losing every backup made with it, so test a restore with it now. The check script below takes it through AGE_IDENTITY.

Retention: delete old dumps with a lifecycle rule

Let the bucket delete old backups, not the script. On S3, a rule that expires everything under the prefix after 35 days:

{
  "Rules": [
    {
      "ID": "expire-old-dumps",
      "Status": "Enabled",
      "Filter": { "Prefix": "postgres/app/" },
      "Expiration": { "Days": 35 }
    }
  ]
}

Applying it replaces the bucket's whole lifecycle configuration, not just this rule, so any expiry or archive rules already on the bucket would silently disappear. Look first:

# prints the current rules, or NoSuchLifecycleConfiguration if there are none
aws s3api get-bucket-lifecycle-configuration --bucket acme-db-backups

# no existing rules, or you merged them into lifecycle.json's Rules array:
aws s3api put-bucket-lifecycle-configuration \
  --bucket acme-db-backups --lifecycle-configuration file://lifecycle.json

The checksum files sit under the same prefix, so they expire with their dumps. If the bucket has versioning turned on, expiring an object only hides it behind a delete marker and the old version is still stored and billed: add a NoncurrentVersionExpiration rule as well, for example { "NoncurrentDays": 7 }. On R2, set the same rule under the bucket's Object Lifecycle Rules. On B2, lifecycle rules first hide a file and then delete it. For more history at little cost, keep dailies for a month and copy the first dump of each month to a second prefix with a longer rule.

The same script on RDS, Neon, Railway, and Supabase

Only the connection string changes. What to use on each platform:

AWS RDS and Aurora PostgreSQL. Connect with TLS verified against the RDS certificate bundle: ?sslmode=verify-full&sslrootcert=/etc/ssl/rds-global-bundle.pem, with the bundle downloaded from truststore.pki.rds.amazonaws.com. RDS's own "export snapshot to S3" feature is not a substitute: it writes Apache Parquet files for analytics tools, which no Postgres can restore. RDS snapshots themselves restore only into RDS. A pg_dump file in your own bucket is the copy you can restore anywhere, including outside AWS.

Neon. Use the direct connection string, the host without -pooler. Neon's own pg_dump guide says pooled connections are not supported for dumps and restores and will cause errors.

Railway. Railway databases are private by default. To dump from outside Railway (a GitHub Actions runner, another server), turn on Public Access under the database's Networking settings, which creates a TCP proxy and a DATABASE_PUBLIC_URL variable to use here. Traffic through that proxy counts as egress on your Railway bill. Running the script as a cron service inside the same Railway project avoids both.

Supabase. Use the Session pooler string on port 5432, and dump your own schemas, not the platform's. Storage files are not in any database dump. The Supabase to S3 guide covers both.

Check that the backup restores

A backup job that has run green for a year proves that pg_dump exited 0 every night. It does not prove the files restore. This second script downloads the newest dump, verifies its checksum, restores it into a throwaway Postgres container of your server's major version, and fails if any table in the archive did not come back. Run it weekly on any machine with Docker, using a separate key that can read and list the bucket, so the backup server's write-only key stays write-only.

#!/usr/bin/env bash
# pg-restore-check.sh: restore the newest dump from the bucket into a throwaway
# Postgres container and check that every table in the archive came back.
# From https://backupdrill.com/guides/pg-dump-to-s3 (MIT licensed).
#
# Needs Docker and the AWS CLI v2. Same variables as pg-dump-to-s3.sh, plus:
#   PG_MAJOR       major version of the pg_dump that made the dumps, which
#                  should be the same as your server's (default: 17)
#   AGE_IDENTITY   path to the age private key, if the dumps are encrypted
#   ROLES          space-separated roles your policies or grants name, e.g.
#                  "app_reader app_writer"; created (NOLOGIN) before restoring
set -euo pipefail

: "${S3_BUCKET:?S3_BUCKET is not set}"
prefix="${S3_PREFIX:-postgres}"
major="${PG_MAJOR:-17}"
container="pg-restore-check-$$"

workdir="$(mktemp -d)"
cleanup() {
  # --volumes: the postgres image keeps its data in an anonymous volume, which
  # would otherwise outlive the container with a plain-text copy of the database
  docker rm --force --volumes "$container" > /dev/null 2>&1 || true
  rm -rf "$workdir"
}
trap cleanup EXIT
umask 077

# 1. Newest dump in the prefix. The timestamped names sort by date.
latest="$(aws s3 ls "s3://$S3_BUCKET/$prefix/" \
  | awk '{print $4}' | grep -E '\.dump(\.age)?$' | sort | tail -n 1 || true)"
[ -n "$latest" ] || { echo "no dumps under s3://$S3_BUCKET/$prefix/"; exit 1; }
aws s3 cp "s3://$S3_BUCKET/$prefix/$latest" "$workdir/$latest" --only-show-errors
aws s3 cp "s3://$S3_BUCKET/$prefix/$latest.sha256" "$workdir/$latest.sha256" --only-show-errors

# 2. Checksum, so a damaged download is not mistaken for a bad backup.
if command -v sha256sum > /dev/null; then
  (cd "$workdir" && sha256sum --check --quiet "$latest.sha256")
else
  (cd "$workdir" && shasum -a 256 --check --quiet "$latest.sha256")
fi

dump="$workdir/latest.dump"
if [ "${latest%.age}" != "$latest" ]; then
  : "${AGE_IDENTITY:?the dump is encrypted: set AGE_IDENTITY}"
  age --decrypt --identity "$AGE_IDENTITY" --output "$dump" "$workdir/$latest"
else
  mv "$workdir/$latest" "$dump"
fi

# 3. A throwaway server of the same major version, with no network at all:
#    it holds a copy of your data, and docker exec is all the checks need.
#    It only listens on TCP once initialisation has finished, so wait for a
#    TCP connection, not pg_isready.
docker run --detach --name "$container" --network none \
  --env POSTGRES_PASSWORD=throwaway "postgres:$major" > /dev/null
for attempt in $(seq 1 120); do
  if docker exec "$container" psql -h 127.0.0.1 -U postgres -c 'select 1' > /dev/null 2>&1; then
    break
  fi
  if [ "$attempt" -eq 120 ]; then
    echo "FAIL: the throwaway Postgres did not start within 2 minutes:"
    docker logs --tail 20 "$container"
    exit 1
  fi
  sleep 1
done
docker cp "$dump" "$container:/tmp/latest.dump" > /dev/null

# --no-owner and --no-privileges drop ownership and grants, but a row-level
# security policy that names a role still needs that role to exist.
for role in ${ROLES:-}; do
  docker exec "$container" psql -h 127.0.0.1 -U postgres -q \
    -c "create role \"$role\" nologin" > /dev/null
done

# 4. Restore. --exit-on-error turns the first failure into a failed check
#    instead of a warning count at the end; --no-tablespaces because the
#    throwaway server has none of the source's custom tablespaces.
docker exec "$container" pg_restore -h 127.0.0.1 -U postgres --dbname=postgres \
  --no-owner --no-privileges --no-tablespaces --exit-on-error /tmp/latest.dump

# 5. Every table that has data in the archive must exist after the restore.
#    A TOC line ends in "TABLE DATA <schema> <table> <owner>"; the table name
#    is everything between schema and owner, so table names with spaces
#    survive. Schema or owner names with spaces are not handled.
docker exec "$container" pg_restore --list /tmp/latest.dump \
  | awk '/ TABLE DATA / {
      sub(/^.* TABLE DATA /, "")
      schema = $1; owner = $NF
      print schema "." substr($0, length(schema) + 2, length($0) - length(schema) - length(owner) - 2)
    }' | sort > "$workdir/expected.txt"
docker exec "$container" psql -h 127.0.0.1 -U postgres -At -c "
  select table_schema || '.' || table_name
  from information_schema.tables
  where table_type = 'BASE TABLE'
    and table_schema not in ('pg_catalog', 'information_schema')" \
  | sort > "$workdir/restored.txt"
missing="$(comm -23 "$workdir/expected.txt" "$workdir/restored.txt")"
if [ -n "$missing" ]; then
  echo "FAIL: tables in the dump but not in the restored database:"
  echo "$missing"
  exit 1
fi

# 6. Row counts, so an empty table stands out.
docker exec "$container" psql -h 127.0.0.1 -U postgres -At -F ' ' -c "
  select table_schema || '.' || table_name,
         (xpath('/row/n/text()', query_to_xml(format('select count(*) as n from %I.%I',
           table_schema, table_name), false, true, '')))[1]::text
  from information_schema.tables
  where table_type = 'BASE TABLE'
    and table_schema not in ('pg_catalog', 'information_schema')
  order by 1"
echo "PASS: $latest restored, $(wc -l < "$workdir/expected.txt" | tr -d ' ') tables present"

A passing run lists every table with its exact row count:

$ bash pg-restore-check.sh
billing.invoices 250
public.customers 300
public.empty_audit 0
public.orders 1000
PASS: 2026-09-29T14-06-30Z.dump restored, 4 tables present

We broke it on purpose before trusting it. A dump truncated in the bucket fails the checksum. A truncated dump with a matching checksum, which is what the piping failure above produces, fails in pg_restore with could not read from input file: end of file. A zero next to a table that should have rows is your cue to look closer. If your row-level security policies name roles, list them in ROLES so the throwaway server has them; otherwise the policies fail to restore. The pg_restore guide linked above explains the flags the script uses and the errors you may meet.

What a year of daily dumps costs on S3, R2, and B2

Say the compressed dump is 2 GB and you keep 30 dailies plus 12 monthlies: 42 copies, about 84 GB stored at any time.

Storage	Per GB-month	84 GB, per month	Reading it back
AWS S3 Standard	$0.023	about $1.93	$0.09/GB after 100 GB/month free
Cloudflare R2	$0.015 (first 10 GB free)	about $1.11	Free
Backblaze B2	$6.95/TB (first 10 GB free)	about $0.51	Free up to 3x what you store

Storage is the small line. The one that grows is reading backups back out: a weekly restore check downloads a full dump each time, which is free on R2, free within B2's allowance, and billed as internet egress on S3 once you pass the free 100 GB a month. Upload requests cost fractions of a cent at one dump a day.

FAQ

Can pg_dump write directly to S3?
No. pg_dump writes to a file or to standard output, and you upload the result. Piping it straight into aws s3 cp - works, but if pg_dump fails halfway the upload still completes and a truncated dump sits in the bucket under a normal-looking name. Dumping to a local file, reading it back in full with pg_restore, and then uploading avoids that at the cost of temporary disk space.

Is pg_dump safe to run while the database is in use?
Yes. pg_dump reads everything from a single snapshot, so the dump is consistent even while writes continue, and it does not block reads or writes. It does take a light lock on each table it dumps, which blocks schema changes such as ALTER TABLE on those tables until the dump finishes. On a large database, schedule it away from migrations.

How often should I back up a PostgreSQL database with pg_dump?
Daily is the usual starting point: a bad day then costs at most a day of data. If that is too much, pg_dump is the wrong tool to run every few minutes; continuous WAL archiving (point-in-time recovery) is the next step. Whatever the schedule, restore-test the dumps regularly, because the schedule only tells you how much you could lose if the backups work.

Sources

Checked 2026-09-29.

This first appeared on backupdrill.com. I build BackupDrill, which does the restore-test part of this for Supabase projects, but everything above is plain Postgres and works without it.

Roblox has three DevEx rates now. One constant won't cut it.

SuperLede — Sun, 27 Sep 2026 08:49:55 +0000

For years every DevEx calculator, spreadsheet and in-game earnings panel I saw did the same thing:

local DEVEX_RATE = 0.0035

That was fine while there was one rate. Since June 2026 there are three, and they apply to the same balance at the same time. If your game shows creators or group members what their Robux are worth, a single constant is now wrong in both directions.

Here's what changed and how I ended up modelling it.

The three rates

Rate	USD per earned Robux	Applies to
Legacy	$0.0035	Robux you already held before Sep 5, 2025
Standard	$0.0038	Robux earned since then
U.S. 18+	$0.0054	Certain sales to US players who verified they're 18+

The standard rate went up on September 5, 2025. Roblox didn't convert old balances at the new rate, though. Anything you held before that date stays at $0.0035, and it cashes out first.

The 18+ rate showed up on June 8, 2026. It only covers developer products, passes, subscriptions and private servers, bought by US players who verified their age, in an experience that meets Roblox's character rules (no R6 avatars). It's a second rate running next to the standard one. It doesn't replace it.

So a real creator balance is often a mix of all three.

Mixed balances

Say someone has 75,000 earned Robux. 20,000 were sitting there before September 2025, 50,000 came in since, and 5,000 came from qualifying 18+ purchases.

With one constant at the new rate you'd tell them $285. With the old one, $262.50. The real number is neither:

local total =
    20000 * 0.0035 -- legacy:    $70
    + 50000 * 0.0038 -- standard: $190
    + 5000 * 0.0054 -- us18:      $27
-- $287

The difference looks small at this size. For a group paying out a few million Robux a quarter it isn't.

Rounding

Roblox pays in USD, so I round to the cent at the end, not per part. Floats make that slightly more interesting than it sounds. 12,345 Robux at $0.0038 is 46.911, so you show $46.91. Going the other way, $100 needs math.ceil(100 / 0.0038) = 26,316 Robux. 26,315 gives 99.997, which rounds to $100.00 on screen but is still under $100 gross.

I don't know exactly how Roblox rounds on its end, so I treat these as gross estimates and say so in the UI.

The 30,000 minimum

You need 30,000 earned Robux for a first cashout. That's $114 at the standard rate. A check is trivial, but it's worth showing players how far they are from it rather than just a dollar figure.

A module instead of a constant

I got tired of retyping this, so I put the rates, the history and the arithmetic into one ModuleScript. No HttpService, no yielding, just a table and five functions:

local DevExRates = require(ReplicatedStorage.DevExRates)

DevExRates.robuxToUsd(100000) --> 380 (standard)
DevExRates.robuxToUsd(30000, "us18") --> 162
DevExRates.usdToRobux(100) --> 26316
DevExRates.rateOn("2024-01-15") --> 0.0035
DevExRates.meetsMinimum(28500) --> false

DevExRates.mixedBalanceUsd({ legacy = 20000, standard = 50000, us18 = 5000 })
--> 287

rateOn returns the standard rate in force on a given date, which is handy if you store historical earnings and want to show what they were worth at the time. It skips the 18+ tier on purpose, since that's a parallel rate and not a change to the standard one.

The code and the JSON behind it are on GitHub: supertj/devex-rates. Code is MIT, data is CC BY 4.0. There's a check script in CI that fails if the numbers in the Luau file drift from the JSON, so the two can't quietly disagree.

Keeping it current

The part that actually breaks things is Roblox changing the rate and nobody updating the constant. I re-check the rates against Roblox's docs at least every 45 days and update the repo when anything moves. The full history with sources is on LootTally's DevEx rates page, which is where the data comes from.

If you only need a one-off number and don't want to touch code, there's also a DevEx calculator that handles the mixed-balance case.

Things I'm still not sure about

How Roblox handles the 10am PT cutoff on September 5, 2025 for Robux that landed that morning. The module treats the whole day as the new rate.
Whether there'll be more tiers. The 18+ rate reads like a first step, not a final shape.

If you've run into either of these with real payouts, I'd like to hear how it went.

pg_restore finished fine. One of my tables wasn't there.

SuperLede — Sun, 27 Sep 2026 07:26:19 +0000

A restore test on a real 105-table Supabase database failed a few weeks ago. The tool doing it restores in a strict mode, so it stopped on the first error: a function with a parameter defaulting to auth.uid(). Fixing that was quick. The interesting part came when I reproduced it locally with plain pg_restore, the way most people test a backup by hand. There it didn't stop at all. It ran to the end, and one table simply wasn't there.

Here's why, how to reproduce it in two minutes, and the few lines that prevent it.

The setup

Supabase gives you auth.uid(), a function that returns the id of the logged-in user. It's everywhere in Supabase schemas, and a very natural place to use it is a column default:

create table public.notes (
  id    serial primary key,
  owner uuid default auth.uid(),
  body  text
);

Insert a row from the client and owner fills itself in. Nice.

Now take a normal backup of your application schema:

pg_dump --format=custom --schema=public -f app.dump "$SUPABASE_DB_URL"

and restore it somewhere that isn't Supabase, which is exactly what you do when you test a backup: a throwaway postgres container.

What goes wrong

Plain Postgres has no auth schema. So when pg_restore gets to CREATE TABLE public.notes (... default auth.uid() ...), the statement fails. The table isn't created. A few lines later the COPY into public.notes fails too, because there's no table to copy into. And then pg_restore keeps going, because by default it reports errors and carries on.

You end up with every other table, all their rows, and no notes.

The error lines are in the output, but they're easy to wave through. Anyone who has restored a Supabase dump into vanilla Postgres has learned to expect a pile of complaints about roles and schemas that "only exist on Supabase", and the tempting rule is to ignore everything mentioning auth. I had written roughly that rule into my own notes. It's wrong for this case.

Reproduce it

Two containers, one pretending to be Supabase:

docker run -d --name src -e POSTGRES_PASSWORD=pw postgres:17
docker run -d --name rt  -e POSTGRES_PASSWORD=pw postgres:17
# wait a few seconds for both to start

docker exec -i src psql -U postgres <<'SQL'
create schema auth;
create function auth.uid() returns uuid language sql stable as $$ select null::uuid $$;
create table public.notes (id serial primary key, owner uuid default auth.uid(), body text);
insert into public.notes (body) select 'n' || g from generate_series(1, 500) g;
create table public.plain (id int);
insert into public.plain select generate_series(1, 100);
SQL

docker exec src pg_dump -U postgres --format=custom --schema=public -f /tmp/app.dump postgres
docker cp src:/tmp/app.dump . && docker cp app.dump rt:/tmp/app.dump

docker exec rt pg_restore -U postgres --no-owner --no-privileges -d postgres /tmp/app.dump
docker exec rt psql -U postgres -c "select to_regclass('public.notes'), (select count(*) from public.plain)"

What I get:

pg_restore: error: could not execute query: ERROR:  schema "auth" does not exist
pg_restore: error: could not execute query: ERROR:  relation "public.notes" does not exist
...
 to_regclass | count
-------------+-------
             |   100

plain restored with all 100 rows. notes doesn't exist.

The same thing happens to a function whose parameter defaults to auth.uid(), like create function is_admin(p_user_id uuid default auth.uid()), which is the one that tripped the strict restore in the first place.

The fix

Before restoring, give the sandbox cheap stand-ins for what Supabase would have provided:

create role anon;
create role authenticated;
create role service_role;
create schema auth;
create function auth.uid()   returns uuid  language sql stable as 'select null::uuid';
create function auth.role()  returns text  language sql stable as 'select null::text';
create function auth.email() returns text  language sql stable as 'select null::text';
create function auth.jwt()   returns jsonb language sql stable as 'select null::jsonb';

Run the same restore again and notes comes back with all 500 rows.

Two details matter here.

First, don't create auth.users. An empty users table would make every foreign key into it fail in a way that looks like your data is broken, when really the sandbox just doesn't have your users. Errors about auth.users are the ones that genuinely are sandbox noise.

Second, change what you ignore. The useful rule isn't "ignore anything mentioning auth". It's: errors about auth.users are expected, and an error saying a role or an auth function doesn't exist means a stub is missing and something didn't restore.

The check that would have caught it anyway

Count things. After any test restore, compare the number of tables and the row counts of your two or three most important tables against production:

select count(*) from information_schema.tables where table_schema = 'public';
select count(*) from public.notes;

A restore that "worked" but is missing a table fails this in five seconds. The exit code won't tell you much here: pg_restore returns non-zero on any error, including the harmless ones every Supabase restore produces, so most people learn to ignore it.

Where this ended up

I build a small backup tool for Supabase, and restore-testing is the part I care most about, so this bug got a proper fix: the drill sandbox now creates these stand-ins before every restore and is strict about which errors count as noise. The CLI is MIT and does the whole drill locally if you want it: github.com/backupdrill/cli.

If you'd rather do it by hand, the step-by-step version is in this restore-testing guide, and a stricter two-pass restore that separates schema-and-data errors from index and constraint errors is in the manual backup guide.

Either way, test the restore, and count what comes back.

I checked 1,000 YouTube channels to see which ones look monetized. Here's what the data says, and what it can't.

SuperLede — Sat, 05 Sep 2026 16:59:36 +0000

In November 2023 YouTube stopped exposing a public flag that said whether a channel was in the Partner Program. Every tool that answered "is this channel monetized" was reading that flag. They all still answer the question. None of them will tell you they're now guessing.

I built one of those tools, so I want to lay out what mine actually reads, where it fails, and what happened when I pointed it at 1,000 channels across 30 countries.

What you can and can't see from outside

"Monetized" means the channel is in the Partner Program. There are two tiers:

Fan funding needs 500 subscribers, 3 public uploads in 90 days, and either 3,000 watch hours in a year or 3M Shorts views in 90 days. It unlocks memberships, Super Thanks and the merch shelf.

Ad revenue needs 1,000 subscribers and either 4,000 watch hours in a year or 10M Shorts views in 90 days.

Watch hours are private. Subscriber count usually isn't. So from outside you can see the eligibility line but not whether the creator applied, got in, or got kicked out later. Anyone quoting an accuracy percentage is quoting a number they have no way to compute.

The five things I read

Every check returns these five as pass, fail or unknown, and the result page shows all five with a sentence each.

Subscriber count, from the official API. It's a clue about eligibility, nothing more. A 40,000-subscriber channel can be unmonetized and a 900-subscriber one can be in the fan funding tier.

The Join button. Memberships are Partner Program only, so if it's there the channel is in. If it's missing that means almost nothing, because plenty of monetized channels never turn memberships on.

Ads on recent long-form uploads. I sample several recent long videos and check whether ads are actually being served. Strongest signal for the ad revenue tier, with a caveat I'll get to.

Merch shelf. Also Partner Program only, also opt-in, also needs a linked store. Present is strong evidence, absent is normal.

The 500-subscriber floor. Below it the channel can't be in the program at all. This is the one signal whose failure is a real negative.

The trap

Since November 2020, YouTube runs ads on videos from channels that are not in the Partner Program. The creator gets nothing from those ads.

So "I saw a pre-roll" proves nothing on its own, and any checker that only looks for ads will confidently call an unmonetized channel monetized. I require ads on at least two thirds of the videos I successfully sampled, and a channel with one lucky pre-roll doesn't clear that.

How the verdict comes out

Evidence has an order. Partner-Program-only features first, then ad delivery, then subscriber counts. Subscriber counts never override a product signal: if a channel shows a Join button at 480 subscribers, the Join button wins.

Four outcomes:

Likely monetized — enough subscribers, memberships on, ads running.
Possibly monetized — one Partner Program feature or the ad signal is there, but the set isn't complete. Usually memberships on, ads not detected.
Likely not monetized — no positive signal at all, and either under 500 subscribers or clearing 1,000 with every readable product signal negative.
Can't tell — the evidence is missing or points both ways. I say so instead of picking.

Two rules that took the longest to get right

Unknown is not fail. A blocked fetch, a hidden subscriber count, a channel with no long-form uploads: none of those are evidence of anything. They stay unknown and never push a verdict toward "not monetized". My first version got this wrong and produced confident false negatives for channels that were merely hard to read.

Failed samples stay out of the denominator. If I try five videos and one loads, "1 of 1 has ads" is not a 100% ad rate, it's a sample of one. Below two successful fetches the ad signal stays unknown.

What 1,000 channels looked like

Guessing about aggregate behaviour is easy, so I measured it. For each of 30 countries I asked the API for the most popular videos right now, took the channels behind them, and checked 35 per country.

Country	Look monetized
Greece, Hungary	51%
Brazil, Romania	49%
France	46%
Germany, Thailand	40%
UK	26%
US	23%
Japan	20%
Korea	9%
India	3%

Two things surprised me. The US is nowhere near the top, despite being the market everyone assumes is most monetized. And the spread is enormous for channels that are all, by construction, big enough to be trending.

I can't fully explain the US number from outside, which is the honest answer. My guess is that US trending skews to music and network content where memberships and merch simply aren't used, but I can't prove that, so it stays a guess in public too.

The other number worth stating: I couldn't tell either way for 18% of channels. That's the number a tool selling certainty would quietly drop.

Full table, method and the CSV are here: https://monetizationkit.com/data/youtube-monetized-channels-by-country/

One engineering note, and a bug worth stealing

The first version read video data from YouTube's internal player endpoint. That's now gated by proof-of-origin tokens for automated callers, so it reads the public watch page instead. Slower, messier, doesn't pretend to be a browser.

The bug I'd point at, because I suspect it's common: I had a rule saying a forced refresh must not overwrite good cached data with a degraded result. Sensible. But it also blocked the very first write for a channel that had never been cached, so every channel whose signals I couldn't read was silently dropped. The dataset could only ever show three of the four verdicts, and it made the tool look more confident than it was. Data that never gets written is much harder to notice than data that's wrong.

The second one, same flavour: my collection run showed ad-fetch failures jumping from 1.2% in the first hour to 30% in the second. I read that as "these channels are unreadable" and nearly published it. It was my own rate limiting. Slowing down and re-running recovered nine in ten. If you scrape for a dataset, plot your failure rate against time before you believe any of it.

What I'd like torn apart

Is there a public signal I'm missing? Super Thanks and Super Chat are Partner Program features too. I surface them where visible but don't score them.
Is one middle tier clearer than two for non-technical creators, or is "possibly" doing too much work?
If you run a channel: does the breakdown match what Studio shows you? That's the only real validation this can get.

The checker is free and needs no login: https://monetizationkit.com/

Supabase Backups Don't Include Your Storage Files. Here's What Does.

SuperLede — Wed, 15 Jul 2026 17:20:39 +0000

The restore finished clean. pg_restore exited 0, every table came back, row counts matched, the app booted. Twenty minutes later the first support message arrived: profile pictures were broken. Then the PDFs. Then every file anyone had ever uploaded.

Nothing about the restore failed. This is how Supabase Storage is built — and it applies equally to the built-in daily backups, to PITR, and to your own pg_dump. None of them include your Storage files.

Your database only stores pointers

Supabase Storage is two systems. The storage.objects table in your Postgres database holds the metadata — bucket, path, timestamps, owner. The actual file bytes live in a separate S3 backend operated by Supabase, entirely outside your database.

Restoring the database restores the pointers. If the files behind them were deleted, or you're restoring into a fresh project, those pointers now reference objects that don't exist. The dangerous part: everything looks healthy from SQL. Row counts match, constraints hold, select count(*) from storage.objects is exactly right — until something requests a file and gets a 404.

Prove it to yourself in five minutes

Ask your database what it thinks is in Storage:

select bucket_id, name, created_at
from storage.objects
order by created_at desc
limit 20;

Now dump your database with pg_dump, restore it into a local Postgres, and run the same query. Every row is there. Not one of the files those rows describe came with the dump. The restore passes every SQL check you can write and still lost your users' data.

Backing the files up

The good news: Supabase exposes the Storage backend over an S3-compatible endpoint, so the files can be copied out like any other bucket.

First, create S3 credentials. Dashboard → Storage → Settings → S3 Access Keys → enable the S3 connection and generate a key pair. The endpoint and region are shown on the same page; the endpoint looks like:

https://<project-ref>.storage.supabase.co/storage/v1/s3

One warning before anything else: these keys bypass RLS. Treat them like a service-role key — server-side only, never in client code.

Option A: DIY with rclone

Sync every bucket to storage you own — Cloudflare R2, Backblaze B2, or your own S3:

# ~/.config/rclone/rclone.conf
[supabase]
type = s3
provider = Other
access_key_id = <SUPABASE_S3_ACCESS_KEY_ID>
secret_access_key = <SUPABASE_S3_SECRET>
endpoint = https://<project-ref>.storage.supabase.co/storage/v1/s3
region = <project-region>

[r2]
type = s3
provider = Cloudflare
access_key_id = <R2_ACCESS_KEY_ID>
secret_access_key = <R2_SECRET>
endpoint = https://<account-id>.r2.cloudflarestorage.com

rclone sync supabase:<bucket> r2:my-backups/storage/<bucket> --checksum

If rclone throws signature errors against the Supabase endpoint (some releases after 1.67.0 did), the plain AWS CLI is a reliable fallback:

aws s3 sync s3://<bucket> ./storage-backup/<bucket> \
  --endpoint-url "https://<project-ref>.storage.supabase.co/storage/v1/s3" \
  --region <project-region>

Two things to get right if you go this route:

Dump the database in the same run. Schedule the rclone sync and the pg_dump together (cron, or a GitHub Actions workflow), so the database snapshot and the file snapshot describe the same moment. Otherwise a restore hands you metadata rows and files that disagree with each other.
Budget for egress. Downloading your files counts as uncached egress on your Supabase bill — $0.09/GB beyond your plan's allowance (Free: 5 GB/month, enforced through a fair-use flow of notice → grace period → restrictions, so recurring large backups aren't viable there; Pro: 250 GB included). For big buckets, weekly beats daily.

Option B: an open-source CLI that does both in one run

Disclosure up front: I'm the author. The backupdrill CLI (MIT, github.com/backupdrill/cli) grew out of exactly the incident at the top of this post. One run streams the pg_dump (custom format) and the Storage files to a bucket you own, and writes a manifest with SHA-256 checksums:

npm install -g backupdrill

export BACKUPDRILL_SUPABASE_STORAGE_ENDPOINT="https://<ref>.storage.supabase.co/storage/v1/s3"
export BACKUPDRILL_SUPABASE_STORAGE_REGION="<project-region>"
export BACKUPDRILL_SUPABASE_STORAGE_ACCESS_KEY_ID="…"
export BACKUPDRILL_SUPABASE_STORAGE_SECRET_ACCESS_KEY="…"

backupdrill backup

You'll also need the database connection string and a destination bucket — the repo covers the full configuration, ships a GitHub Actions template for scheduling, and has an estimate command that projects the egress cost before you commit to a schedule.

The part I actually built it for is backupdrill drill: it restores the latest snapshot into a throwaway Docker Postgres and verifies it — archive checksum, table count against the manifest, populated tables restored non-empty, sampled SHA-256 checks on the Storage files — then destroys the container. Because a backup you've never restored is a guess.

Option C: the hosted version

If you'd rather not babysit cron: BackupDrill runs the same open-source engine on a schedule — database and Storage files captured together into your own bucket, restore drills that prove the snapshot actually comes back, failures emailed. The free plan is one project with weekly backups and includes a restore drill on your first backup, no card.

The takeaway

Whatever you use for the database — built-in backups, PITR at $100 per month, or your own dumps — none of it covers Storage files. If users upload anything you'd mind losing, the files need their own backup path, and the database snapshot and file snapshot need to describe the same moment.

This is an expanded version of a guide originally published on backupdrill.com. I build BackupDrill; the CLI above is MIT and works without the hosted service.