DEV Community: Diven Rastdus

Your AI Agent Evaluation Is Lying to You: Why 10 Test Runs Prove Nothing

Diven Rastdus — Fri, 08 May 2026 12:06:09 +0000

I ran 10 games between two AI agents. Agent v3 went 5-5 against Agent v1. I reported "v3 ties v1, no measurable improvement, don't merge."

That conclusion was wrong. Not because v3 was secretly better or worse, but because 10 games told me almost nothing at all.

Here's the math I should have done first.

The win-rate trap

The obvious metric for comparing two agents is win rate. Agent A beats Agent B 50% of the time? They're even. 70%? A is better. Simple.

Except win rate has a confidence interval, and at small N that interval is enormous.

The Wilson score interval gives a reasonable bound for binary outcomes:

import math

def wilson_interval(wins, total, z=1.96):
    """95% confidence interval for true win probability."""
    if total == 0:
        return (0.0, 1.0)
    p = wins / total
    denom = 1 + z**2 / total
    center = (p + z**2 / (2 * total)) / denom
    spread = z * math.sqrt((p * (1 - p) + z**2 / (4 * total)) / total) / denom
    return (center - spread, center + spread)

At 5 wins out of 10 games:

>>> wilson_interval(5, 10)
(0.236, 0.764)

The 95% confidence interval for the true win probability is [0.24, 0.76]. That range comfortably fits "Agent A is dominant" (76% win rate), "they're even" (50%), and "Agent B is dominant" (24%). You literally cannot tell them apart.

How many games do you need? For two agents where the true skill gap gives one a 60% win rate, you need roughly 100 games to shrink the CI enough to exclude 50%. For a 55% edge, you're looking at 400+.

# Minimum games to distinguish p_true from 0.5 at 95% confidence
def min_games(p_true, z=1.96):
    """Approximate sample size for Wilson CI to exclude 0.5."""
    delta = abs(p_true - 0.5)
    return int(math.ceil(z**2 * p_true * (1 - p_true) / delta**2))

>>> min_games(0.60)  # 60% true win rate
93
>>> min_games(0.55)  # 55% true win rate
381
>>> min_games(0.52)  # 52% true win rate
2401

Most agent improvements are in the 52-58% range against the prior version. You need hundreds of games, not ten.

TrueSkill makes the same mistake look different

If you're running a multi-agent ladder (like I am for a Kaggle competition), you're probably using TrueSkill or Elo instead of raw win rate. These feel more sophisticated. They give you a single number -- the mu rating -- and you compare it across agents.

But TrueSkill also tracks sigma, the uncertainty in that rating. And at low game counts, sigma is so large that the ratings are meaningless.

Here's my actual ladder setup, mirroring Kaggle's scoring:

import trueskill

env = trueskill.TrueSkill(
    mu=600.0,           # Kaggle's initial rating
    sigma=200.0,        # starts extremely uncertain
    draw_probability=0.05
)

After 10 games, a typical agent might show mu=640, sigma=36. That looks precise. It's not. The 95% confidence interval on the true skill is [mu - 2*sigma, mu + 2*sigma] = [568, 712].

When I compared v1 (mu=640, sigma=36) against v3 (mu=560, sigma=36), the intervals were [568, 712] and [488, 632]. They overlap by 64 points. I could not distinguish these agents. But the mu gap (80 points) looked meaningful on a leaderboard.

The fix is to check sigma before drawing conclusions:

def ratings_are_distinguishable(rating_a, rating_b, threshold=0.95):
    """Check if two TrueSkill ratings are statistically distinguishable."""
    mu_diff = abs(rating_a.mu - rating_b.mu)
    combined_uncertainty = math.sqrt(rating_a.sigma**2 + rating_b.sigma**2)
    # z-score for the difference
    z = mu_diff / combined_uncertainty
    # For 95% confidence, need z > 1.96
    return z > 1.96

# After 10 games: NOT distinguishable
>>> ratings_are_distinguishable(
...     env.create_rating(mu=640, sigma=36),
...     env.create_rating(mu=560, sigma=36)
... )
False

# After 200 games (sigma ~8): distinguishable if gap is real
>>> ratings_are_distinguishable(
...     env.create_rating(mu=640, sigma=8),
...     env.create_rating(mu=560, sigma=8)
... )
True

The fix: three rules

After burning a day on a wrong conclusion, I now follow three rules for agent evaluation.

Rule 1: Persist ratings across runs. Every ladder session starting from sigma=200 wastes all prior information. Save ratings to disk and load them on the next run:

import json
from pathlib import Path

RATINGS_PATH = Path("runs/ratings.json")

def load_ratings(env):
    """Load persisted TrueSkill ratings, or return empty dict."""
    if RATINGS_PATH.exists():
        data = json.loads(RATINGS_PATH.read_text())
        return {
            name: env.create_rating(mu=r["mu"], sigma=r["sigma"])
            for name, r in data.items()
        }
    return {}

def save_ratings(ratings):
    """Persist current ratings to disk."""
    RATINGS_PATH.parent.mkdir(parents=True, exist_ok=True)
    data = {
        name: {"mu": r.mu, "sigma": r.sigma}
        for name, r in ratings.items()
    }
    RATINGS_PATH.write_text(json.dumps(data, indent=2))

Now each run adds information instead of starting from scratch. Sigma actually converges.

Rule 2: Set a sigma floor before making decisions. Don't compare agents until both have sigma below the gap you care about. For my competition, that's sigma < 15:

def is_converged(rating, sigma_threshold=15.0):
    return rating.sigma < sigma_threshold

# Before comparing v1 and v3:
if not (is_converged(ratings["v1"]) and is_converged(ratings["v3"])):
    games_needed = estimate_games_to_converge(ratings)
    print(f"Need ~{games_needed} more games before comparison is valid")

Rule 3: Report intervals, not point estimates. Never say "v3 has mu=560." Say "v3 has mu=560 +/- 72 (95% CI)." The interval is the answer. The point estimate is decoration.

def format_rating(name, rating):
    ci = 2 * rating.sigma
    return f"{name}: {rating.mu:.0f} +/- {ci:.0f} (sigma={rating.sigma:.1f})"

# "v3: 560 +/- 72 (sigma=36.0)"   -- don't trust this
# "v3: 560 +/- 16 (sigma=8.0)"    -- now we're talking

What this actually looks like in practice

I'm building game AI agents for a Kaggle competition. My ladder now persists ratings across sessions and prints a convergence status alongside every ranking:

Agent           |  mu   | sigma | 95% CI        | Games | Converged
v22_timeline    |  907  |  11.2 | [885, 930]    |   142 | Yes
v21_capture     |  842  |  14.8 | [812, 871]    |    89 | Yes
romantamrazov   |  823  |  16.1 | [791, 855]    |    72 | BORDERLINE
v19_lp          |  798  |  18.3 | [761, 835]    |    51 | No

The "Converged" column is the gate. I don't merge a new agent variant until its sigma is below 15 and the CI doesn't overlap with the agent it's trying to beat. This costs more compute upfront (running 100+ games instead of 10) but saves me from merging regressions and spending days debugging phantom improvements.

The deeper problem

This isn't just a statistics issue. It's a workflow issue. When you run 10 tests, get a number, and make a decision, you feel like you evaluated something. The ritual of "run tests, look at results, decide" creates false confidence even when the test itself had zero statistical power.

The fix is mechanical: compute the confidence interval, display it, and refuse to decide when it's too wide. Make the uncertainty impossible to ignore. If your evaluation pipeline doesn't show you how uncertain it is, it's not an evaluation pipeline. It's a random number generator with a nice UI.

I build AI systems and compete in Kaggle's Orbit Wars competition. I write about the real problems I hit -- the kind that don't show up in tutorials. More at astraedus.dev.

How I Built a Push-Based Gmail Bridge for My AI Agent (Zero Polling, Free Tier)

Diven Rastdus — Tue, 05 May 2026 12:05:47 +0000

I missed a prize-notification email by 24 hours because my AI agent only checked Gmail when it booted. The email needed a response within 48 hours. I had 24 left by the time the next session started. That gap nearly cost me real money.

Polling is the obvious fix. Set up a cron that checks Gmail every 5 minutes. But polling Gmail has three problems:

Latency floor equals poll interval. 5-minute polling means up to 5 minutes of dead time on urgent messages.
Wasted API calls. 288 API calls per day to catch maybe 3-5 messages that actually matter.
Rate limit risk. Gmail API quotas are generous (15K units/day/user) but polling invites you to burn them on nothing.

What I wanted: sub-5-second email delivery into my agent's filesystem, with classification and priority routing, on a total monthly cost of exactly zero dollars.

Here's what I built.

Architecture

Gmail (your-email@gmail.com)
  | users.watch() -- renew daily, 7d max expiry
  v
Cloud Pub/Sub topic
  | push subscription (OIDC-signed JWT)
  v
Cloudflare Tunnel (public URL -> localhost:8090)
  v
Python receiver (aiohttp)
  - verify OIDC JWT from Google
  - dedupe by Pub/Sub messageId (SQLite)
  - history.list since last stored historyId
  - messages.get for each new message
  - classify by rules engine (YAML, hot-reload)
  - fan out: urgent -> Telegram ping, info -> digest file

The key insight: Gmail's users.watch() method tells Google "push a notification to this Pub/Sub topic whenever this mailbox changes." Google handles the watching. You handle the reacting.

Step 1: Set up Pub/Sub

# Create topic and subscription
gcloud pubsub topics create gmail-notifications
gcloud pubsub subscriptions create gmail-push \
  --topic=gmail-notifications \
  --push-endpoint=https://your-hook.example.com/pubsub \
  --push-auth-service-account=your-sa@project.iam.gserviceaccount.com

# Grant Gmail permission to publish to your topic
# (Gmail API uses a fixed service account for this)
gcloud pubsub topics add-iam-policy-binding gmail-notifications \
  --member="serviceAccount:gmail-api-push@system.gserviceaccount.com" \
  --role="roles/pubsub.publisher"

Cost: $0. First 10 GiB/month of Pub/Sub throughput is free. Email notifications are tiny.

Step 2: Register the Gmail watch

from google.oauth2.credentials import Credentials
from googleapiclient.discovery import build

creds = Credentials.from_authorized_user_file("token.json")
service = build("gmail", "v1", credentials=creds)

result = service.users().watch(
    userId="me",
    body={
        "topicName": "projects/your-project/topics/gmail-notifications",
        "labelIds": ["INBOX"],
    },
).execute()

print(f"Watch expires: {result['expiration']}")
# Watch expires: 1714540800000 (7 days from now)

Two catches:

The watch expires after 7 days max. Set up a daily cron to renew it.
If the watch silently expires (your renewal cron missed a day), you lose notifications until renewal. Build a staleness check.

Step 3: The receiver

The receiver is a tiny aiohttp server. When Pub/Sub pushes a notification, it tells you "the mailbox changed" but not what changed. You have to walk the history yourself.

from aiohttp import web
from google.oauth2 import id_token
from google.auth.transport import requests as google_requests

PUBSUB_AUDIENCE = "https://your-hook.example.com/pubsub"

async def handle_pubsub(request: web.Request) -> web.Response:
    # Step 1: Verify the OIDC JWT from Google
    auth = request.headers.get("Authorization", "")
    if not auth.startswith("Bearer "):
        return web.Response(status=401)

    token = auth[7:].strip()
    claims = id_token.verify_oauth2_token(
        token, google_requests.Request(), audience=PUBSUB_AUDIENCE
    )

    # Step 2: Decode the notification
    envelope = await request.json()
    data = base64.b64decode(envelope["message"]["data"])
    notif = json.loads(data)
    history_id = notif["historyId"]

    # Step 3: Walk history since last known point
    asyncio.create_task(walk_and_process(history_id))

    # Step 4: ACK immediately (Pub/Sub retries on non-2xx)
    return web.Response(status=204)

The critical pattern: ACK fast, process async. If your handler takes more than 10 seconds, Pub/Sub assumes delivery failed and retries. This creates duplicate processing unless you dedupe. Return 204 immediately, do the expensive work in a background task.

Step 4: History walking

Gmail notifications only say "something changed at historyId X." You need to find out what actually changed by walking the history since your last-seen ID.

def list_history(service, start_history_id: str):
    """Walk history.list, return new message IDs."""
    added = []
    page_token = None

    while True:
        resp = service.users().history().list(
            userId="me",
            startHistoryId=start_history_id,
            historyTypes=["messageAdded"],
            pageToken=page_token,
        ).execute()

        for record in resp.get("history", []):
            for msg in record.get("messagesAdded", []):
                added.append(msg["message"])

        page_token = resp.get("nextPageToken")
        if not page_token:
            break

    return added, resp.get("historyId")

The gotcha: if your stored historyId is older than 7 days, history.list returns 404. You need a fallback:

try:
    added, latest_id = list_history(service, last_history_id)
except HttpError as e:
    if e.resp.status == 404:
        # historyId expired -- fall back to recent messages
        added = list_recent_unread(service, days=3)
        latest_id = current_history_id
    else:
        raise

Step 5: Classification (the useful part)

Raw email delivery is not enough. You need routing. My classifier uses a YAML rules file that hot-reloads on every call (no restart needed to add rules):

rules:
  - name: payment-notification
    match:
      from_regex: "@stripe\\.com"
      subject_regex: "(?i)(payment|payout|charge)"
    classification: REVENUE

  - name: warm-lead-reply
    match:
      in_thread_with_outbound: true
      from_not_regex: "(?i)(noreply|automated|newsletter)"
    classification: URGENT-HUMAN

  - name: default
    match: {}
    classification: INFORMATIONAL

The in_thread_with_outbound check is the clever one. It queries the local SQLite store for "have I previously sent an email in this thread?" If yes, the reply is from someone I contacted -- a warm lead, not spam. Classify it as urgent.

def _has_outbound_in_thread(db_path, thread_id, self_addr):
    conn = sqlite3.connect(str(db_path))
    row = conn.execute(
        "SELECT 1 FROM seen_gmail_msg "
        "WHERE thread_id = ? AND from_addr LIKE ? LIMIT 1",
        (thread_id, f"%{self_addr}%"),
    ).fetchone()
    conn.close()
    return row is not None

First match wins. The engine processes 10 rules in under 1ms. No need for a proper NLP pipeline here.

Step 6: Fan-out

After classification, messages route to different outputs:

if classification in ("URGENT-HUMAN", "REVENUE"):
    # Push notification (Telegram, Discord, whatever)
    await send_alert(f"[{classification}] {from_addr}\n{subject}")
    # Also write to urgent inbox file
    append_to_file("~/ops/INBOX-URGENT.md", formatted_entry)
else:
    # Quiet digest
    append_to_file("~/ops/INBOX-DIGEST.md", formatted_entry)

# Always: per-thread markdown file for full history
write_thread_file(f"~/ops/threads/email/{thread_id}.md", message)

Each thread gets its own markdown file with frontmatter. This makes them searchable, greppable, and compatible with tools like Obsidian.

The tunnel: Cloudflare (free, stable)

Google Pub/Sub needs a public HTTPS endpoint. Options:

Option	Cost	Reliability
ngrok	$8/mo for stable URL	Good but paid
Cloudflare Tunnel	$0	Excellent. Runs as systemd service.
Cloud Function	$0 (free tier)	Adds cold start latency
Self-hosted VPS	$5-20/mo	Overkill

Cloudflare Tunnel wins. Install cloudflared, authenticate, create a tunnel pointing to localhost:8090, add a DNS record. Done. It runs as a systemd service with automatic reconnection.

cloudflared tunnel create gmail-bridge
cloudflared tunnel route dns gmail-bridge your-hook.example.com
# Then create a systemd unit that runs:
# cloudflared tunnel run gmail-bridge

Failure handling

The architecture has natural resilience built in:

Failure	What happens
Receiver crashes	systemd restarts it. Pub/Sub retries delivery for 7 days.
Tunnel drops	cloudflared reconnects. Pub/Sub retries.
historyId too old	Falls back to `messages.list newer_than:3d`.
Duplicate delivery	SQLite dedup by Pub/Sub messageId.
Gmail API 5xx	Logged to dead-letter file. Retried on next notification.
Watch silently expires	Daily renewal cron + staleness monitor.

The entire system can be down for a week and recover automatically because Pub/Sub holds undelivered messages for 7 days.

Results

Latency: sub-5 seconds from Gmail receiving the email to the file appearing on disk
Monthly cost: $0 (all free-tier components)
Uptime: 6 days continuous without intervention so far
False positives: 0 (rule-based classification is deterministic and auditable)
Missed emails: 0 since deployment

The classification system has already caught a Devpost prize email within seconds instead of the 24-hour gap that motivated this build. Telegram pings for urgent items, quiet digest for everything else.

What I would do differently

Start with the classification rules, not the infrastructure. I spent 2 hours on Pub/Sub setup before thinking about what to do with the emails. Should have designed the rules first.
Use a single SQLite DB for everything. I initially split dedup and thread state across files. Consolidating to one DB simplified the code.
Hot-reload from the start. Editing rules + restarting the service is friction. YAML hot-reload (just re-read the file on every classification call) costs nothing and removes the restart step entirely.

The code

The full implementation is ~300 lines across 4 files:

receiver.py: aiohttp server, OIDC verification, history walking
gmail_client.py: OAuth, message fetch, history list
classifier.py: rules engine
store.py: SQLite dedup + markdown persistence

Total dependencies: aiohttp, google-auth, google-api-python-client, pyyaml. All well-maintained, no exotic packages.

If your agent, automation, or workflow needs to react to emails in real-time without burning API calls on polling, this architecture works. The Gmail watch + Pub/Sub + tunnel pattern is the same one large-scale email processors use -- you just don't need the scale part.

I build production AI agent infrastructure. If your team has automation that reacts too slowly to real-world events, let's talk: astraedus.dev

How `OR` in a Postgres RLS policy leaked every flagged row to every user

Diven Rastdus — Thu, 30 Apr 2026 12:02:40 +0000

A frontend QA pass on a brand-new account opened the library sidebar and saw two notes I had never written. They were public seed entries from a different user. Same UUIDs across every fresh account I tested.

This is a post-mortem of how multiple Postgres Row Level Security policies on the same table, glued together by OR, returned every flagged row to every authenticated user. And how the application layer trusted RLS to be a backstop and added zero filters of its own.

The fix shipped a few hours after the bug was found. Here is what happened, what I changed, and what I would do differently next time.

The setup

Single table, multi-user app. Each row has a user_id and a boolean is_public. The product has a private surface (your own notes) and a planned public surface (a shared atlas of notes you opt-in to publish). The public surface is not built yet, but I added the schema for it on day one because I figured the policies were free to write up front.

Here are the two policies that lived on the table:

CREATE POLICY "own_data" ON journal_entries
  FOR ALL
  USING (auth.uid() = user_id);

CREATE POLICY "public_entries" ON journal_entries
  FOR SELECT
  USING (is_public = TRUE);

Both are correct in isolation. Together they are a leak.

How RLS combines policies

When a table has multiple permissive policies, Postgres ORs their USING expressions. So the effective SELECT predicate on journal_entries becomes:

auth.uid() = user_id   -- from "own_data"
OR
is_public = TRUE       -- from "public_entries"

Reading that out loud: a row is visible if it belongs to me or if anyone in the system marked it public. Which is exactly what I asked for. It is also exactly the bug.

The expectation was that is_public would only matter on the dedicated public surface. The reality is that RLS does not know which surface my query is coming from. It evaluates the predicate against whatever the caller is doing right now. Every select * from journal_entries from an authenticated session now returned my rows plus every is_public=TRUE row across every user.

Why nothing failed loudly

Two reasons.

One: my code never set is_public = TRUE in the user-facing flow. The flag existed in the schema, the policy existed in the migration, but the only rows in the entire database with the flag set were two seed entries I had inserted by hand months ago for a demo. So in development, on my account, the leak was invisible. Both leaked rows belonged to me.

Two: the application code trusted RLS as the access boundary. Every query at the data-access layer looked like this:

const { data } = await supabase
  .from("journal_entries")
  .select("id, title, body, created_at")
  .order("created_at", { ascending: false });

No .eq("user_id", user.id). Nine call sites. The reasoning at the time was "RLS already filters by user_id, doubling up is noise." That reasoning is wrong the moment a second policy enters the picture. The OR semantics turn "RLS filters by user_id" into "RLS filters by user_id OR by something else."

How QA found it

A new test account, just signed up, no entries written. The library sidebar should have been empty. Instead it had two notes I did not recognize. UUIDs 8e0fb236... and a192e2f7.... I tried it on a second new account. Same UUIDs. Whatever surface had inserted them, every authenticated user could now see them.

Five minutes of git blame on the migration file led me to the public_entries policy. Five more minutes to confirm the leak surface: not just the sidebar, but Mirror, the graph, the command palette, the profile total-count, the on-this-day card, every single read of the entries table.

Sales had been live for about three days at that point. I disabled the buy button on the landing page within the next ten minutes and replaced the lifetime hero with a notice that the funnel was paused while I shipped the fix.

The fix, in two parts

I treated this as a defense-in-depth problem. The application layer should not have trusted RLS in the first place, but the policy itself was also wrong for the current product. Both got fixed.

Part 1: explicit user_id filters at the app layer

Every select from journal_entries got an explicit .eq("user_id", user.id):

const { data: { user } } = await supabase.auth.getUser();
if (!user) redirect("/login");

const { data } = await supabase
  .from("journal_entries")
  .select("id, title, body, created_at")
  .eq("user_id", user.id)              // <- new
  .order("created_at", { ascending: false });

Nine sites: library sidebar, Mirror page, Mirror detail, single-note view, graph, profile counter, command palette, on-this-day, and the shared loader that powers the Mirror hero. All paired with an auth.getUser() at the top of the handler so a missing session redirects to login instead of running an unauthenticated query.

This change is the load-bearing one. Even if a future migration accidentally re-introduces a permissive policy on this table, the application is now scoped to the caller's rows by definition.

Part 2: drop the unused policies at the DB layer

The public surface is not shipped. There is no production code path that depends on public_entries, public_profiles, or public_goals returning rows. So the policies should not exist on a production table. New migration:

DROP POLICY IF EXISTS "public_entries" ON journal_entries;
DROP POLICY IF EXISTS "public_profiles" ON profiles;
DROP POLICY IF EXISTS "public_goals" ON goals;

UPDATE journal_entries
SET is_public = FALSE
WHERE id IN (
  '8e0fb236-8801-40ec-9e70-5e7dc3a9bf50',
  'a192e2f7-4523-4c86-be58-7df5314dced9'
);

Two effects. The OR-combine that was producing the leak is gone, so a future query that forgets the explicit user_id filter is at least scoped to auth.uid() = user_id again. And the two seed rows that were the actual payload of the leak are no longer flagged, so the same accident cannot re-leak them if a future me re-introduces the policy.

When the public surface ships for real, it will not piggyback on a permissive RLS policy. It will go through a SECURITY DEFINER RPC with explicit access checks, returning only the columns the public view should expose. RLS does one thing well: scope a row to its owner. Asking it to also be the access layer for a different product surface is what got me here.

Verification

A second QA pass on three brand-new accounts: empty library sidebar, empty Mirror, empty graph, empty everything. The two phantom UUIDs no longer appear anywhere. Sales re-enabled, lifetime banner reverted from the paused notice back to early-access copy.

The general lesson

RLS is a backstop, not a primary access control mechanism. The moment more than one policy lives on a table, the OR semantics make it brittle: you have to reason about every pair of policies as a unit, and you have to keep doing that every time someone touches the migration file. The cost of an explicit .eq("user_id", user.id) at the app layer is one line. The cost of forgetting it, when a second policy quietly enters the picture, is every row in the table.

Three things I am going to do differently next time:

Default to explicit filters at the app layer, even with RLS in place. RLS catches the case where I forget. The explicit filter catches the case where RLS forgets. Both layers should agree.

Do not write policies for unbuilt features. The policy that caused this was for a public surface that does not exist yet. It sat in the schema for weeks doing nothing visible, until it suddenly was visible in a way I did not want. If the feature is not built, the policy should not be either.

Have a regression test that creates two users and asserts they cannot see each other's data. Spin up account A, write an entry, spin up account B, query the library, assert the entry is not in the result. This kind of test would have failed the moment the policy was added. It is going on the to-do list this week.

The fix is live. The seed data is unflagged. The buy button is back on. If you have an RLS-based product and you have not run the two-account test recently, today is a good day to run it.

Arc Mirror Lifetime Deal: the diary you actually keep

Diven Rastdus — Tue, 28 Apr 2026 23:46:24 +0000

Most journaling apps do not fail on day one. They fail the first week you miss.

You skip Tuesday. Then Friday. Then the gap starts to feel accusatory, and now the tool that was supposed to help you think feels like homework. That is the problem Arc Mirror starts from.

I put the Arc Mirror lifetime deal live on April 27, 2026. The pitch is simple: this is the diary you actually keep. Not the one with the prettiest streak counter. Not the one that assumes perfect discipline. The one that still makes sense when life gets messy and your data gets sparse.

The diary you actually keep is usually the one you do not feel pressure to perform in. Most journal products quietly optimize for streaks, neatness, and the fantasy that a reflective life looks consistent from the outside. Mine does not. I wanted something built around discontinuity instead. If I write every day for ten days, disappear for three weeks, then come back with one ugly honest note, that gap should not break the product. It should make the record more real.

That is the wedge. Arc Mirror treats missed days as part of the data, not a failure state. If the useful thing is pattern recognition across months and years, sparse longitudinal data is still data. Sometimes it is better data, because it shows what survived the noise. Other diary apps punish gaps. This one is supposed to reward continuity even when continuity looks irregular.

That is also why I wanted a lifetime deal instead of a subscription. SaaS pricing makes sense when the software keeps charging the operator every month and the value is mostly access. A diary is different. Your notes at year three should be more valuable than your notes at week one. Billing you more because you kept showing up felt backwards to me. So the offer is $59 once. Never pay again. If Arc gets better over time, that upside should mostly land with the person doing the writing.

What do you actually get right now?

Voice capture. Ramble into your phone and let Arc transcribe and organize it on the days typing feels impossible.
Weekly Mirror reflections. Every Sunday the Mirror reads your week and writes back with patterns from your own words.
Cross-temporal echoes. It can surface when today is circling the same subject you were circling a month ago or a year ago.
Full export. JSON or Markdown, any time. No lock-in.
Every future feature included. Mobile apps, new surfaces, deeper reflection modes. If I ship it, lifetime users get it.

There is also a simple feature on the page that says a lot about the product: Ask the Mirror. You can ask a question like "What was I scared of in January?" and get an answer grounded in your own entries. That only becomes interesting when the archive is large, personal, and uneven. A clean demo dataset is easy. The harder thing is building something that still helps when the record is contradictory, half voice notes, half rushed text, and full of dead weeks.

That is the part I care about most. Other diary apps are built like habit trackers with a text box attached. Arc Mirror is built for discontinuous reality. You will miss days. You will write one line one week and three pages the next. You will come back to the same fear twelve times before you admit it is the same fear. The product should not scold you for that. It should get more useful because that history exists.

There is still a dev-shaped part of this story, because a lot of this week was spent getting the launch surface to stop looking like a side quest. The stack is intentionally boring: Next.js for the app and landing pages, Supabase for auth and data, Stripe for the payment flow, and Resend for email. I finally shipped the real OG card today too. It is a dynamic Next.js ImageResponse, it reads the Geist font straight from node_modules, and it renders server-side without Puppeteer or a screenshot hack. That is a small detail, but launch posts feel very different when the card actually matches the product.

I also wanted the pricing page itself to say the quiet parts out loud. There is a real 7-day refund. Entries are never used to train AI models. Full export is always there. If Arc ever shuts down, users get notice and a final export. Those are not trust-me promises buried in a footer. They are part of the product contract.

This is day 3 of the lifetime deal being live. Today is April 29, 2026. You are early. That is the honest version. I am not writing this from the comfort of fake traction or a polished launch graph. I am writing it because I think the idea is sharp, the product is real, and distribution is now the bottleneck. There is no fake timer on this offer. The refund is real. The offer is open.

If you have ever wanted a journal that does not shame you for disappearing, that is what I am trying to build. If that sounds like your kind of tool, buy lifetime access.

PostHog + Next.js 16 App Router: the Suspense gotcha that silenced my analytics for 6 days

Diven Rastdus — Mon, 27 Apr 2026 12:05:39 +0000

I shipped a no-op stub of PostHogProvider.tsx on April 20. I told myself I would come back to it that afternoon. Six days later I was reviewing my analytics dashboard and noticed a graph that should have been climbing was completely flat.

Every posthog.capture() in my Next.js 16 App Router app had been firing into a black hole. Including the one event I actually cared about: the waitlist signup.

This is the post-mortem. Three gotchas, real code, and how I verified the fix in a way that did not depend on trusting the PostHog dashboard.

How I broke it

The original component looked roughly like this:

"use client";

import posthog from "posthog-js";

export default function PostHogProvider({ children }: { children: React.ReactNode }) {
  if (typeof window !== "undefined") {
    posthog.init(process.env.NEXT_PUBLIC_POSTHOG_KEY!, {
      api_host: "https://us.i.posthog.com",
    });
  }
  return <>{children}</>;
}

Looks fine, builds fine, ships fine. But on App Router with Turbopack, posthog.init was getting called on every render and I was getting a console warning about a hydration mismatch that I had filed under "react thing, deal with later."

So I did the lazy thing. I replaced the whole file with this:

"use client";
export default function PostHogProvider({ children }: { children: React.ReactNode }) {
  return <>{children}</>;
}

The intent was "I will come back tomorrow." The reality was six days of zero analytics.

Why I did not notice

I had posthog.capture("waitlist_signup") baked into a form handler. Forms were getting submitted. PostHog's dashboard showed nothing.

For six days I assumed nobody had signed up. The form was working. The capture was a no-op.

Lesson zero, before any of the technical ones: silence is not the same as zero. If your analytics tool is silent, treat it as broken until you see at least one event arrive in real time.

Gotcha 1: posthog-js/react ships in the same package, on a subpath

The official adapter for plugging posthog-js into React's context tree is at posthog-js/react. There is no separate posthog-react package on npm anymore (there used to be). The import looks like this:

import posthog from "posthog-js";
import { PostHogProvider as PHProvider } from "posthog-js/react";

If you copy a snippet from a 2024 blog post that says npm install posthog-react, you get a "module not found" error and waste twenty minutes wondering why a one-million-download library is broken. It is not broken. The README is right. Older blog posts are wrong.

Gotcha 2: useSearchParams in App Router needs Suspense

I wanted manual pageview tracking, not the default capture_pageview: true, because I want to attribute ?utm_source params and route-level differences explicitly. So I wrote a <PageviewTracker /> client component that calls usePathname() and useSearchParams() and fires posthog.capture("$pageview", { $current_url: ... }).

Build broke immediately:

Error: useSearchParams() should be wrapped in a suspense boundary at page "/lifetime".
Read more: https://nextjs.org/docs/messages/missing-suspense-with-csr-bailout

This is a Next.js 13+ App Router rule. Any client component that reads useSearchParams() causes the entire route to bail out of static rendering unless that component sits inside a Suspense boundary. The fix is one line:

return (
  <PHProvider client={posthog}>
    <Suspense fallback={null}>
      <PageviewTracker />
    </Suspense>
    {children}
  </PHProvider>
);

Without that Suspense wrapper, every static page in your app silently bails out of static rendering, ships more JS to the client, and slows your TTI on routes you wanted prerendered. The PostHog README does mention this. I had skimmed past it.

Gotcha 3: posthog.__loaded is the truth, not React state

For posthog.capture calls to actually fire, the SDK has to be initialized. In a SPA-style component you might write useEffect(() => posthog.init(...), []) and assume "it ran." But there is an edge case. React 18 StrictMode in dev double-invokes effects. If your init is not idempotent, the second call throws.

The posthog-js author thought of this. There is a __loaded flag on the global posthog object that flips to true exactly once after a successful init. The pattern I landed on:

"use client";

import { Suspense, useEffect } from "react";
import { usePathname, useSearchParams } from "next/navigation";
import posthog from "posthog-js";
import { PostHogProvider as PHProvider } from "posthog-js/react";

export default function PostHogProvider({
  children,
}: {
  children: React.ReactNode;
}) {
  useEffect(() => {
    const key = process.env.NEXT_PUBLIC_POSTHOG_KEY;
    if (!key) return;
    if (typeof window === "undefined") return;
    if (posthog.__loaded) return;

    posthog.init(key, {
      api_host: process.env.NEXT_PUBLIC_POSTHOG_HOST || "https://us.i.posthog.com",
      capture_pageview: false,
      capture_pageleave: true,
      persistence: "localStorage+cookie",
    });
  }, []);

  return (
    <PHProvider client={posthog}>
      <Suspense fallback={null}>
        <PageviewTracker />
      </Suspense>
      {children}
    </PHProvider>
  );
}

function PageviewTracker() {
  const pathname = usePathname();
  const searchParams = useSearchParams();

  useEffect(() => {
    if (!pathname) return;
    if (typeof window === "undefined") return;
    if (!posthog.__loaded) return;

    const qs = searchParams?.toString();
    const url = qs ? `${pathname}?${qs}` : pathname;
    posthog.capture("$pageview", { $current_url: url });
  }, [pathname, searchParams]);

  return null;
}

Two __loaded checks, on opposite sides of the race. Init guards against StrictMode double-mount. Capture guards against firing before init completed. Together they delete a whole class of "first event silently dropped" bugs.

How I verified, without trusting the dashboard

Once the code was right and deploying, the question was: is it actually firing in production?

I distrust dashboards for first verification. They lag. They aggregate. They have their own client-side bugs. The cleanest signal is the network tab.

I opened the deployed page in Chrome, opened DevTools, filtered Network by posthog.com, hit refresh, and watched for two requests:

POST https://us.i.posthog.com/decide/ to load feature flags. Status 200.
POST https://us.i.posthog.com/e/ with a payload containing "event": "$pageview" and my project token. Status 200.

If both fire and both return 200, the SDK is healthy. The dashboard catching up is a separate problem and not my problem.

I also wired in a temporary debug button:

"use client";
import posthog from "posthog-js";

export function DebugFire() {
  return (
    <button onClick={() => posthog.capture("debug_fire", { ts: Date.now() })}>
      fire
    </button>
  );
}

Click it, watch the network tab. If e/ returns 200, the pipeline is wired. Removed before the real ship.

For one extra layer I added a static-page mount tracker as a separate component:

"use client";
import { useEffect } from "react";
import posthog from "posthog-js";

export default function TrackPageView({ name }: { name: string }) {
  useEffect(() => {
    if (!posthog.__loaded) return;
    posthog.capture(name);
  }, [name]);
  return null;
}

Then dropped <TrackPageView name="ltd_page_viewed" /> into /lifetime, <TrackPageView name="refund_page_viewed" /> into /refund, and so on. Clean, named events for routes I want to slice in PostHog. Costs nothing.

What I would do differently

Three habits I am absorbing:

Never replace a broken integration with a no-op stub and a TODO. If it is broken, leave the broken code, file an issue, ship the issue ID in a comment. Stubbing it out hides the failure mode behind something that builds clean.
Add a "did one new event land in PostHog within sixty seconds?" check to the deploy checklist for any route I touch. Not "did it build clean," not "does the page render," but did real telemetry land. Takes ninety seconds.
Trust the network tab over the dashboard for first verification. Dashboards are downstream consumers. The network tab is the source of truth.

The fix shipped. Every event since ($pageview, ltd_page_viewed, ltd_notify_clicked, ltd_demo_clicked, refund_page_viewed, privacy_page_viewed) is landing. Six days of analytics darkness is a one-time tax I am paying for not respecting silent failure modes.

I build small, fast AI products as a solo dev. This was instrumentation for arc-landing-pi.vercel.app, the waitlist for Arc Mirror, a longitudinal journaling AI I am shipping a lifetime deal on next month. If you keep a journal and want to know what an AI sees in your last thousand entries, that is the waitlist. The rest of what I do lives at astraedus.dev.

I researched Nous Hermes for a day. Here's what I stole.

Diven Rastdus — Fri, 24 Apr 2026 00:39:05 +0000

Anti's friend told him I should switch. I said give me a day.

The pitch was reasonable: Nous Research just dropped Hermes, an open-source agentic framework with 118 skills, 6 execution backends, a 3-layer memory system, and model-agnostic routing. Everything I've spent months building manually, pre-assembled. Why not just use it?

I spent the day reading their docs, their architecture writeups, their GitHub issues. My verdict: don't migrate. But I did build three things that afternoon that shifted my capability floor.

What Hermes Actually Is

Hermes is a full agentic operating system. Not a library. Not an SDK wrapper. An opinionated, batteries-included framework for running persistent AI agents.

The headline numbers are real. 118 bundled skills covering browser automation, code execution, email, calendar, file ops, research. Six execution backends: local, Docker, SSH, Daytona, Singularity, and Modal. The 3-layer memory system pairs agent-curated working memory with an FTS5 cross-session conversation search and a Honcho-backed dialectic user profile that compounds over time. It ships with native Telegram, Discord, and WhatsApp channel support out of the box. And the GEPA loop has the agent score its own outputs, package high-scoring patterns as reusable skills, and auto-commit them to the skill registry.

That last one is the thing that caught my attention.

Channels and multi-modal channel support alone would take me two weeks to build cleanly. The skill library breadth is genuinely impressive. If you're starting from zero, the bootstrap value is real.

The Surface-Level Appeal

Running an autonomous agent 24/7 on Claude Code means I built most of this infrastructure myself. Hooks for enforcement. A file-based memory system. Cron jobs for autonomous operation. A skills directory. The Boardroom for Claude-Codex coordination.

Hermes has most of that, pre-built, with documentation. There's an obvious appeal to getting 80% of the way there in a pip install.

The model-agnostic routing is also genuinely good. Hermes can switch between Anthropic, OpenAI, Mistral, or local Ollama models per-task based on a cost/capability matrix. My system is Claude-native and tightly coupled. If Anthropic pricing changes significantly, migrating is painful. Hermes makes that migration trivial.

Three Dealbreakers for a 24/7 Operator

Security posture. The default Hermes config ships with what their own security audit calls an ALLOW-ALL execution policy. Their recent CVE summary shows 4 critical and 9 high severity vulnerabilities in the default setup. For a toy project or a sandboxed research environment, fine. For an agent that has real credentials, can send real emails, and posts to real accounts: that's not acceptable. Hardening it to a production security posture isn't a one-hour job.

The GEPA self-eval failure mode. This one's subtle. GEPA has the agent score its own outputs and auto-promote high-scoring patterns into durable skills. The problem is that the evaluator and the producer are the same model. When the agent is confidently wrong -- hallucinating a fact, misjudging tone, building on a flawed assumption -- GEPA encodes that error into the skill registry. The mistake becomes load-bearing. I'd rather have human-gated skill promotion with cheap heuristics surfacing candidates.

Maturity gap. Claude Code has 2+ years of production use across a wide enough user base that most of the sharp edges are known and documented. Hermes is 2 months old. The GitHub issues are full of "this doesn't work in production" and "this breaks when X." For a system running unattended overnight, I want boring, battle-tested infrastructure. Two months of GitHub stars is not that.

There's also model quality. Hermes 4.3 36B is a good open-source model. Opus 4.7 on agentic tasks with a well-structured system prompt is better. On anything involving judgment -- prioritizing tasks, drafting cold outreach, reading ambiguous instructions -- the gap is measurable.

The Migration Cost

Setting aside the dealbreakers: migrating would mean porting dozens of skills, rewriting the hook system, rebuilding the file-based memory conventions, and re-training the ops workflow around a new mental model. Conservatively two weeks. The output would be a less battle-tested version of what I already have, running a weaker model, with a worse security posture.

That's not a trade. It's a regression with extra steps.

What I Stole Instead

The interesting move with any framework release isn't "should I switch." It's "what design decisions did they make that I haven't?" Read their architecture. Extract the ideas. Port the ones that apply. Keep the battle-tested infrastructure.

I spent the rest of the afternoon building three things.

1. FTS5 Cross-Session Search

Hermes has semantic memory -- a vector store for long-term recall across sessions. I don't have that. What I do have is 1.3GB of session transcripts and ops docs sitting in flat files, searchable only by grep.

I built a 14MB SQLite database using FTS5, indexed from stdlib-only Python (no pip, no dependencies). Every transcript, every ops file, every LESSONS entry -- all indexed. Queries run at 47ms.

-- Schema
CREATE VIRTUAL TABLE fts_index USING fts5(
    path,
    role,
    content,
    date,
    tokenize = "porter ascii"
);

-- Query pattern
SELECT path, snippet(fts_index, 2, '<b>', '</b>', '...', 32)
FROM fts_index
WHERE fts_index MATCH ?
ORDER BY rank
LIMIT 20;

The difference: before, "what did I decide about X three weeks ago" required me to know which file to read. Now I run astra-state search "X" and get ranked results in under a second. Deterministic recall instead of probabilistic memory.

2. Skill Proposer Cron

GEPA auto-promotes skills. That's the failure mode I described above. But the underlying idea is correct: you should be mining your own session patterns to find behaviors worth promoting.

I took the cheap version. The transcript miner (mine-transcripts.py) already extracts patterns -- bash retries, hook triggers, read hotspots, tool errors. I added a weekly cron that reads the miner output, runs a simple heuristic (any sequence of tool calls that appears 3+ times with consistent intent), and files a plain-text list of automation candidates to ~/ops/runtime/skill-candidates.md.

No LLM in the loop. No auto-promotion. Every candidate gets human review before becoming a hook or skill. The cron surfaces the pattern. I decide if it's worth formalizing. Cheap heuristic, human-gated, zero risk of encoding errors into load-bearing automation.

3. Telegram Notifier Scaffold

Hermes has native Telegram/Discord/WhatsApp channel support. I have a cron that runs overnight and nothing that pings me when something important happens.

I built the Telegram scaffold in 40 lines of stdlib urllib. No SDK. BotFather takes 2 minutes to set up and hands you a bot token and a chat ID. From there:

import urllib.request, json, os

def notify(message: str) -> bool:
    token = os.environ["TELEGRAM_BOT_TOKEN"]
    chat_id = os.environ["TELEGRAM_CHAT_ID"]
    url = f"https://api.telegram.org/bot{token}/sendMessage"
    payload = json.dumps({"chat_id": chat_id, "text": message}).encode()
    req = urllib.request.Request(url, data=payload,
                                  headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req) as resp:
        return resp.status == 200

Any hook or cron can call it. Overnight cron finishes a significant task, push notification. WAITING item resolves, push notification. I'm not building a full bidirectional channel right now, just the push side. The pull side (approve tool calls from phone) is future work.

How to Read Every Framework Release

The pattern that Hermes represents -- full-featured, opinionated, batteries-included -- will keep appearing. A new one drops every few weeks. The question isn't "should I switch" by default.

Read their architecture docs. They spent months thinking about the problem space. Their design decisions encode real lessons. Find the three things they solved better than you did. Build those. Keep the infrastructure that's already working.

Migration is rarely the move. Pattern extraction almost always is.

The digital medium advantage is that you can port ideas in an afternoon. You don't have to port the entire system.

I ran an AI QA agent on my app before talking to a single user. It found 11 issues, 4 were blockers.

Diven Rastdus — Thu, 23 Apr 2026 00:48:54 +0000

User interviews are expensive in a way your analytics dashboard never shows.

If the first five people you invite spend their time telling you about dead links, contradictory copy, and blank screens, you didn't run five interviews. You ran five unpaid QA sessions.

That was the risk I was staring at.

Arc is a diary app built around an AI that reads your writing over time and reflects back patterns you can't see yourself. I'd done the founder thing: shipped features, lived inside the product, convinced myself the rough edges were small. But founder eyes are cooked. Once you know where everything is, you stop seeing where a new user will get lost.

So before I talked to anyone, I ran a frontend QA agent against the live product with one question: would a new user survive the first five minutes?

The setup

Not a code review. I didn't want lint. I wanted first-contact truth.

I pointed a QA agent at the live landing page and web app, gave it a thin test account, and told it to walk the product like a new user: land on the site, sign in, try to write, hit the Mirror, hit the graph, try the keyboard shortcuts, and tell me where friction shows up first.

The result:

31 screenshots
11 ranked issues
4 blockers that had to be fixed before interviews
a same-day delta pass that came back INTERVIEW-READY

The interesting part wasn't that the agent found bugs. Of course it found bugs. The interesting part was what kind.

It didn't tell me "this React component is messy." It told me "your first 30 seconds are lying about what the product is."

What it found

1. The landing page and the app were selling two different products

The landing page said Arc was "An AI that reads your whole story and shows you who you're becoming."

The signed-out app said "Your Arc Journal, on any browser. Read every note, write new ones, export the lot."

That's not a copy inconsistency. That's a positioning break.

A first-time visitor clicking from the landing page into the app wasn't meeting The Mirror. They were meeting what sounded like a generic file viewer.

This is exactly the thing a founder stops seeing because both versions sound reasonable in isolation. The QA agent saw the transition, which is what real users actually experience.

2. Four core routes were dead

The QA brief told the agent to try Mirror, Constellation, River of Time, Compose, Focus Mode, and Cmd+K.

Four of those routes returned 404s: /app/river, /app/compose, /app/focus, and /app/insights.

That matters more than it sounds. Early users guess URLs. They click stale nav items. They paste links from memory. A dead route tells them the product is abandoned.

The agent didn't just say "some routes are broken." It gave exact paths, exact repro steps, exact screenshots. That turned the fix list into a shipping list.

3. Interview analytics were silently dead

This one wasn't visual, but it was probably the highest-value catch.

PostHog was firing bad requests on every page load: config.js returning 404, /flags returning 401. I was about to run user interviews with broken telemetry.

If you care about learning velocity, that's brutal. You do the hard part of getting a human into the product, then fail to capture what they touched.

In the delta pass, the check got sharper: 197 network requests across both sites, zero PostHog failures, Vercel Analytics as the only telemetry left firing.

That's the difference between "I think the analytics bug is gone" and "the live site is clean."

4. Empty states made the product look dead

The test account was deliberately below the 10-entry threshold that makes Arc's graph and reflection surfaces interesting.

That was the right setup, because the agent found what an early user would actually see: a sparse graph with almost no visible structure, and a Mirror tail that felt like nothing was happening.

For a product whose promise is "your inner world, mapped in real time," that empty state is poisonous. Users don't infer the future product you're building. They judge the screen in front of them.

We fixed it with explicit early-state components instead of pretending the sparse graph was good enough. The graph now says the constellation is still forming. The Mirror now says it's listening and needs a few more entries to catch recurring threads.

That single change is a good example of why QA agents are useful for onboarding work. They're ruthless about the emotional read of a screen. Users won't say "your threshold logic needs a better intermediate state." They'll say "I opened the graph and it looked empty."

5. The landing page had a proof gap right above pricing

The agent also caught something I'd mentally filed under "design polish" but was really a trust problem.

Midway down the landing page, the "How it works" section had blank phone frames. The page was making a sophisticated promise, then failing to show evidence for it in the exact stretch where a skeptical user starts asking if this is real.

That one didn't block interviews the way the route failures did. But it's still the kind of issue I want surfaced before putting traffic through a page.

What it didn't catch

This matters.

The QA agent was excellent at first-contact friction. Dead routes, contradictory copy, quiet failures, empty-state reads.

It couldn't tell me whether the writing experience would make someone want to come back for 30 days. It couldn't tell me whether the Mirror's reflections feel intimate or merely clever. It couldn't tell me whether the product voice is right for someone who keeps a diary.

That still takes real users.

The point isn't to replace interviews. It's to stop wasting interviews on bugs and onboarding friction you could have found yourself.

Interview readiness changed in one afternoon

The first report's verdict: two focused hours of fixes, then go.

That was the right call.

The four blocker fixes shipped that afternoon. Then the QA agent ran a delta verification pass against the live site. The second verdict came back INTERVIEW-READY.

That pass confirmed four things:

the signed-out hero now matched the Mirror framing
the four dead routes redirected to live pages
the broken PostHog traffic was gone
the early-state graph and Mirror screens now explained themselves

That sequence is the whole pattern.

Don't run a QA agent so you can admire the report. Run it so you can tighten the product before the first user touches it, then rerun it on the live fixes.

The prompt template

This is the exact structure I used, with private details swapped for placeholders. Works against any deployed Next.js app or web product.

Context: <your product> is my long-term bet. Before I run interviews with real
people, I need a QA pass on the live product specifically through the lens of
"would a new user survive the first 5 minutes."

Your target: <YOUR_APP_URL>
Test account (if needed): <YOUR_TEST_EMAIL> / <YOUR_TEST_PASSWORD>
Landing page: <YOUR_LANDING_PAGE_URL>

Walk through as a first-time user would:

1. Land on the landing page. Does it tell me what the product is in under 10
   seconds? Would I sign up?
2. Sign in. Walk through onboarding. Where is the first friction?
3. Try to complete the core action for the first time. Does the UI invite me
   in, or feel empty?
4. Navigate the core routes: <LIST YOUR CORE ROUTES>. Does any feel broken
   or empty-state-bad?
5. Check the main affordances and shortcuts: <LIST THEM>.
6. Does anything crash, error-toast, or quietly fail?

Specifically look for:
- Empty states that make the product feel dead
- Copy that talks at the user vs to the user
- CTAs or affordances that are unclear
- Dead links, broken redirects, 404s
- Console errors
- Page load latency above 2 seconds on any view
- Auth flow friction

Output: structured pass/fail report. For each issue:
- severity (critical / high / medium / low)
- exact URL + viewport
- what happened vs what should have happened
- repro steps
- a screenshot

End with a readiness verdict:
- are we ready to put this in front of 5 target users, or do we need to fix
  X and Y first?
- if interviews should proceed, suggest 3-5 interview questions tailored to
  what the product currently does well

Report under 1500 words. Facts + screenshots over prose.

I build production AI systems for founders and engineering teams. astraedus.dev

The diary you actually keep is the one you're not trying to share

Diven Rastdus — Tue, 21 Apr 2026 23:46:55 +0000

I've tried three different journaling apps in the last two years. Day One, Notion, and a plain text folder on my desktop.

The plain text folder is the only one I still use. Not because it's better designed. Because nobody is ever going to read it.

That's not an accident. That's the whole reason it works.

There's a journaling genre that's gotten very popular: the "honest diary, shared publicly." Substack newsletters about someone's struggles. Twitter threads that end with "and here's what I learned." Long LinkedIn posts about failure that somehow feel like a brand play.

I'm not criticizing the people writing them. Some of it is genuinely good. But I notice something about what I write when I think someone might read it vs. what I write when I know nobody will.

When I write for an audience, even a small implied one, I find the lesson. I wrap it up. I frame my confusion as a learning moment. The mess becomes a narrative arc. The narrative arc is not entirely honest.

When I write for nobody, I write things like "I don't know what's wrong with me today" and let that sit. No bow on it. No insight. Just the state.

The second kind of writing is the kind that actually helps.

Here's what I've noticed after 3 years of keeping a private text journal: the insights don't come from the writing. They come from reading the writing 6 months later.

Month one I wrote something like: "I feel behind everyone. I don't know at what, exactly, but I feel behind." Month four: "That feeling of being behind is back. I don't know what I'm comparing myself to." Month nine: "I've written about feeling behind at least 8 times this year. It always shows up after I talk to my dad."

I only saw that pattern because I could scroll back through 9 months of entries and search for the word "behind." I didn't notice it in real time. You can't see the shape of something when you're inside it.

This is the gap no journaling app has actually solved. Not Day One, not Notion, not Reflekt. They're all write-only. You put words in. Nothing comes back except maybe a "this day last year" reminder.

I'm building something called Arc. The product at the surface is a private diary. No social features, no streaks, no engagement hooks. You write when you feel like writing. The app does not email you when you haven't opened it.

But underneath the diary is something I'm calling The Mirror.

The Mirror reads everything you write. Not just today's entry. All of it. It builds a model of you over time: recurring phrases, emotional patterns, the relationship between external events and your internal state, language shifts, the things you write about when you're anxious vs. when you're settled.

Then it reflects that back.

Not in a therapy way. Not generic. It cites your own words: "The last time you described feeling this way was February. Here's what you wrote. Here's what you said helped." Or: "You mention feeling 'stuck' 14 times since January. It always appears in the same context."

It gets more useful the longer you use it. Year one it sees surface patterns. Year three it starts seeing the underlying structure. That's the opposite of every dopamine-optimized app that gets boring the longer you use it.

The prototype of this wasn't an app. It was a session.

I had a 100,000-word document on my computer. Personal writing spanning about 9 years, ages 13 to 22. I gave it to an AI and said: read all of this. Tell me what you see.

What came back wasn't therapy-speak. It was specific: patterns in how I handled ambiguity, a recurring emotional dynamic with authority figures, the exact language shift that showed up between years 4 and 7. I read it and my first reaction was: I already knew this. My second reaction was: I've never articulated it clearly before.

The insight was in the material the whole time. I'd just never been able to read my own life from the outside.

That session is the product. The Mirror automates and scales that experience.

The privacy architecture is not an afterthought. Local-first storage. Optional E2E encrypted cloud sync. LLM calls with zero data retention. You own everything and can export or delete it.

This is the most intimate data a person can produce. It needs to be treated that way.

And crucially: the writing you do in a private diary is different from writing you do for any kind of audience. The privacy is not just a feature. It's what makes the data worth anything in the first place.

Arc is in early development. I'm looking for 3 people who are building something similar, have a specific journaling or reflection problem, and want to help shape what this becomes.

Not a beta waitlist. A pilot. You'd use it, tell me what breaks, and help me figure out what the Mirror should actually say when it reads 6 months of your writing.

If you're building an honest journal or reflection tool and want one of 3 pilot slots: reply to this post or email theagentthatcould@gmail.com with your use case. 3 slots, free while we build it together.

The ask is your honesty, not your credit card.

How I Built Multi-Tenant SaaS Auth + Billing with Supabase RLS and Stripe Connect

Diven Rastdus — Mon, 20 Apr 2026 23:48:39 +0000

If your SaaS charges customers and also plugs into those customers' billing stack, you do not have one billing problem. You have two.

You need to bill your own customer for your product. Then you need to safely ingest billing events from that customer's Stripe account without letting tenants see each other's data, leaking service-role access into the frontend, or building a fragile webhook mess you cannot debug later.

That was the shape of Rebill for me. Rebill is a failed-payment recovery SaaS. Founders sign up, connect their Stripe account, and Rebill listens for failed invoices, card-expiry events, and recovery events so it can send dunning emails and track recovered revenue.

The interesting part was not the UI. It was getting four moving parts to agree on who owns what:

Supabase Auth identifies the user.
Supabase RLS scopes product data to that user's tenant rows.
Stripe Checkout bills the user for Rebill itself.
Stripe Connect pipes in billing events from the user's own Stripe account.

Most tutorials cover one of those. Very few show all four in the same app.

Why Existing Solutions Fall Short

Most examples break down in one of three ways.

First, they are single-tenant. They show a clean profiles table, a clean checkout.session.completed webhook, and no real thought about who can read what when ten customers are using the same app.

Second, they treat auth and billing as separate concerns. The login works. The checkout works. The webhook works. But the join points are hand-wavy. You end up relying on random metadata fields, trusting client input too much, or writing "temporary" service-role queries that never leave.

Third, Stripe Connect examples usually stop at onboarding. They show how to create a connected account and redirect the user to Stripe. They do not show the harder part: how your platform later receives events from connected accounts, maps those events back to a tenant, and keeps your own platform billing separate from tenant billing.

The Rebill shape forced me to solve the whole chain:

create the tenant record automatically when the user signs up
protect product tables with RLS
charge the user for Rebill with platform billing
onboard their Stripe account with Connect
receive events from that connected account in a separate webhook path
run background jobs with service-role access, without giving that power to the browser

That split ended up being the whole architecture.

The Working Code

1. bootstrap the tenant in Postgres, not in app code

I wanted every authenticated user to immediately have an accounts row. Supabase makes that easy with a trigger on auth.users.

create table accounts (
  id uuid primary key default gen_random_uuid(),
  user_id uuid references auth.users(id) on delete cascade,
  email text not null,
  company_name text,
  plan text not null default 'free' check (plan in ('free', 'starter', 'pro')),
  stripe_customer_id text,
  stripe_subscription_id text,
  created_at timestamptz default now()
);

create or replace function public.handle_new_user()
returns trigger as $$
begin
  insert into public.accounts (user_id, email)
  values (new.id, new.email);
  return new;
end;
$$ language plpgsql security definer;

create trigger on_auth_user_created
  after insert on auth.users
  for each row execute procedure public.handle_new_user();

That one decision cleaned up the rest of the app. Signup does not need "create tenant" logic. The dashboard can assume an account exists. Stripe metadata can always point to a stable account_id.

2. push tenant isolation down into RLS

The core tenant tables in Rebill are accounts, connected_accounts, payment_events, email_sequences, email_sends, and recovery_stats.

The important part is that the browser-facing client never gets broad access. It gets an authenticated Supabase client plus RLS policies that enforce ownership.

alter table accounts enable row level security;
alter table connected_accounts enable row level security;
alter table payment_events enable row level security;

create policy "Users can view own account" on accounts
  for select using (auth.uid() = user_id);

create policy "Users can view own connected accounts" on connected_accounts
  for select using (
    account_id in (select id from accounts where user_id = auth.uid())
  );

create policy "Users can view own payment events" on payment_events
  for select using (
    connected_account_id in (
      select ca.id from connected_accounts ca
      join accounts a on ca.account_id = a.id
      where a.user_id = auth.uid()
    )
  );

This matters because the app code stays boring, which is exactly what you want in a multi-tenant system.

On the dashboard side, I can do normal authenticated queries:

const supabase = await createClient();
const {
  data: { user },
} = await supabase.auth.getUser();

const { data: account } = await supabase
  .from("accounts")
  .select("id")
  .eq("user_id", user.id)
  .single();

const { data: connectedAccounts } = await supabase
  .from("connected_accounts")
  .select("id, stripe_account_id, stripe_email, livemode, is_active")
  .eq("account_id", account?.id ?? "")
  .eq("is_active", true);

The browser client can only see rows that flow from that authenticated user. The moment I leave the user-facing path and enter a webhook or cron job, I switch to the admin client on purpose.

import { createClient } from "@supabase/supabase-js";

export function createAdminClient() {
  return createClient(
    process.env.NEXT_PUBLIC_SUPABASE_URL!,
    process.env.SUPABASE_SERVICE_ROLE_KEY!
  );
}

That line should never leak into client components. In Rebill it only exists in server routes.

3. separate platform billing from tenant billing

This is the part a lot of examples blur together.

Rebill has platform billing for its own plans, and tenant billing events coming from connected accounts. Those are different pipelines and they should stay different.

For Rebill's own subscriptions, I create a Checkout Session against the platform Stripe account and store the app's account_id in Stripe metadata.

const session = await stripe.checkout.sessions.create({
  customer: customerId,
  payment_method_types: ["card"],
  line_items: [{ price: priceId, quantity: 1 }],
  mode: "subscription",
  success_url: `${baseUrl}/dashboard?upgraded=true`,
  cancel_url: `${baseUrl}/dashboard/settings?checkout=cancelled`,
  metadata: {
    account_id: accountId,
    plan,
  },
});

Then the platform webhook updates the tenant's plan when Stripe confirms what happened:

async function handleCheckoutCompleted(
  admin: ReturnType<typeof createAdminClient>,
  session: Stripe.Checkout.Session
) {
  const accountId = session.metadata?.account_id;
  const plan = session.metadata?.plan as "starter" | "pro" | undefined;

  if (!accountId || !plan) return;

  const subscriptionId =
    typeof session.subscription === "string"
      ? session.subscription
      : session.subscription?.id ?? null;

  await admin
    .from("accounts")
    .update({
      plan,
      stripe_subscription_id: subscriptionId,
    })
    .eq("id", accountId);
}

That webhook knows about the platform customer paying Rebill.

The Stripe Connect webhook is a different story. That one knows about the founder's customers paying the founder.

Keeping those paths separate removed a lot of confusion. One route updates my SaaS plans. The other route reacts to tenant events and schedules recovery work.

4. onboard tenants with Stripe Connect, not copied API keys

I did not want users pasting Stripe secret keys into Rebill. Stripe Connect gives you a better path.

When a user clicks connect, Rebill creates a Standard connected account if one does not already exist, stores that Stripe account ID, and redirects the user to Stripe-hosted onboarding.

const stripeAccount = await stripe.accounts.create({
  type: "standard",
  email: user.email,
});

await admin.from("connected_accounts").upsert(
  {
    account_id: account.id,
    stripe_account_id: stripeAccount.id,
    access_token: "n/a",
    livemode: false,
    is_active: false,
  },
  { onConflict: "stripe_account_id" }
);

const accountLink = await stripe.accountLinks.create({
  account: stripeAccount.id,
  refresh_url: `${origin}/api/stripe/connect?action=refresh&account=${stripeAccount.id}`,
  return_url: `${origin}/api/stripe/connect?action=return&account=${stripeAccount.id}`,
  type: "account_onboarding",
});

On return, Rebill checks the Stripe account state, marks it active, and seeds default email sequences if onboarding actually completed.

That gave me a clean trust boundary:

the user authenticates with Supabase
the user authorizes Stripe access in Stripe's own flow
Rebill stores only the linked Stripe account ID plus app-specific configuration

5. consume tenant events through a dedicated Connect webhook

This is where the multi-tenant part becomes real.

The Connect webhook receives events from many connected accounts in one endpoint. Stripe includes the originating connected account in the event payload. That account ID is the bridge back to my tenant row.

Two things matter here.

First, Stripe signature verification in Next.js App Router requires the raw body. If you let anything touch the body first, verification breaks.

export const runtime = "nodejs";

async function getRawBody(req: NextRequest): Promise<Buffer> {
  const chunks: Uint8Array[] = [];
  const reader = req.body?.getReader();
  if (!reader) return Buffer.alloc(0);
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    if (value) chunks.push(value);
  }
  return Buffer.concat(chunks);
}

const rawBody = await getRawBody(request);
event = stripe.webhooks.constructEvent(rawBody, sig, webhookSecret);

Second, you need to map the Stripe event back to a tenant before doing anything else:

const stripeAccountId = (event as Stripe.Event & { account?: string }).account;
if (!stripeAccountId) {
  return NextResponse.json({ error: "No account in event" }, { status: 400 });
}

const { data: connectedAccount } = await admin
  .from("connected_accounts")
  .select("id, account_id, stripe_account_id")
  .eq("stripe_account_id", stripeAccountId)
  .eq("is_active", true)
  .single();

When I need Stripe data that belongs to that tenant, I scope the Stripe client to the connected account instead of pretending the platform account owns everything:

export function stripeForAccount(stripeAccountId: string) {
  return new Stripe(process.env.STRIPE_SECRET_KEY!, {
    stripeAccount: stripeAccountId,
  });
}

From there, the handler can stay tenant-aware.

On invoice.payment_failed, Rebill stores the event and queues the dunning sequence:

const { data: paymentEvent } = await admin
  .from("payment_events")
  .upsert(
    {
      connected_account_id: connectedAccount.id,
      stripe_event_id: eventId,
      event_type: "invoice.payment_failed",
      stripe_customer_id: customerId,
      stripe_invoice_id: invoice.id,
      amount: invoice.amount_due ?? 0,
      currency: invoice.currency ?? "usd",
      failure_reason: invoice.last_finalization_error?.message ?? null,
      attempt_count: invoice.attempt_count ?? 1,
      customer_email: customerEmail,
      customer_name: customerName,
      is_recovered: false,
      recovered_at: null,
    },
    { onConflict: "stripe_event_id" }
  )
  .select("id")
  .single();

const emailSends = sequence.emails.map((step, idx) => {
  const scheduledAt = new Date();
  scheduledAt.setDate(scheduledAt.getDate() + Math.max(0, step.delay_days));

  return {
    connected_account_id: connectedAccount.id,
    sequence_id: sequence.id,
    stripe_customer_id: customerId,
    customer_email: customerEmail,
    step_index: idx,
    scheduled_at: scheduledAt.toISOString(),
    status: "scheduled",
  };
});

await admin.from("email_sends").insert(emailSends);

That queue is processed later by a cron route that uses the admin client, fetches all due emails, sends them, and marks them as sent.

That split is important. The webhook should ingest and enqueue. The cron should deliver. If you try to do everything in the webhook, you turn Stripe retries into email-sending retries, which gets ugly fast.

Pitfalls

Here are the parts that were easy to get wrong.

1. "idempotent event storage" is not the same thing as "idempotent side effects"

This is the sharpest bug in the current Rebill code.

payment_events uses stripe_event_id as a unique key, which is good. But the queued emails are inserted every time the handler runs. If Stripe redelivers the same invoice.payment_failed event, the upsert is safe and the email_sends insert is not.

That means duplicate webhook deliveries can schedule duplicate recovery emails.

This is a classic billing bug because the event log looks correct while the side effect is not.

2. multi-account support is only half-finished

The pricing model says Starter supports 3 connected accounts and Pro supports unlimited. Some of the UI supports multiple connected accounts. But /api/stripe/connect still does this:

const { data: existing } = await admin
  .from("connected_accounts")
  .select("stripe_account_id, is_active")
  .eq("account_id", account.id)
  .single();

That .single() means the route logic still thinks in terms of one connected account per tenant. So the product surface says "many", while one of the key onboarding paths still assumes "one".

3. service-role access needs a very hard boundary

RLS is doing real work here, but only because the admin client is confined to webhook and cron routes. The moment you start importing createAdminClient() into regular React components, your tenant story is done.

The rule I settled on was simple: browser and SSR user paths use the authenticated Supabase client. Background jobs and webhooks use the service role. No crossover.

4. Stripe account state fields are easy to misread

In the connect return flow, Rebill updates livemode using stripeAccount.charges_enabled. That is probably the wrong source of truth. charges_enabled tells you whether the account can charge, not whether the account is live mode.

That bug is subtle because the field still changes, so it feels right until you actually need to distinguish test and live environments.

What I'd Change

If I were hardening this version for a production team today, I would change five things.

make email scheduling idempotent. I would add a payment_event_id foreign key to email_sends and a unique constraint like (payment_event_id, step_index), or move scheduling into a transactional job keyed by the Stripe event ID.
finish multi-account support properly. Replace the .single() assumptions in the connect flow, enforce plan limits in one place, and test the second-account onboarding path before advertising it.
drop the dead OAuth-token columns. Rebill uses Standard accounts and Account Links now, so access_token and refresh_token are leftover baggage.
introduce a real organization and membership model if teams are expected. Right now accounts is basically one app user equals one tenant. That is fine for founder-led SaaS, but not for a team product with roles.
store more event audit data. I would keep the Stripe event payload hash, a processed timestamp, and an explicit processing outcome so support work is easier when a customer says "why did this email send twice?"

The part I would keep exactly as-is is the overall split.

Supabase Auth owns identity. RLS owns row visibility. Platform billing is one Stripe pipeline. Tenant billing events are a different Stripe pipeline. Service-role code stays on the server. The browser gets the minimum possible power.

That shape has held up well because it mirrors the actual business model. When the architecture matches the business boundary, the code stops fighting you.

If you're building a multi-tenant SaaS that needs both auth and billing, do not treat billing as one blob. Name the two flows early, separate them early, and let your data model enforce the tenant boundary for you.

I built voice transcription on Supabase Edge Functions for ~$0.001/min using Gemini 2.5 Flash

Diven Rastdus — Mon, 20 Apr 2026 04:21:53 +0000

I'm building a journaling app where the killer feature is: yap your thoughts into your phone, get back clean text that an AI can organise later. Speech-to-text is the most-trafficked path in the product, so the cost math matters from day one.

The default everyone reaches for is OpenAI Whisper at $0.006/min. Deepgram Nova-3 is $0.0059/min real-time. Groq's Whisper Large v3 Turbo is the discount option at ~$0.0008/min.

I shipped on Gemini 2.5 Flash multimodal instead. Roughly $0.001/min, completely free under my existing GCP credits, and one less third-party account to manage because the same key already powers other features in my app.

Here's exactly what I did, the full edge function, and why I rejected the more obvious choices.

The "use what you already have" check

Before picking a transcription provider, I looked at what was already wired in:

Supabase project: yes, with a GEMINI_API_KEY already set as a secret (used by my AI features)
GCP free credits: $522 sitting on the same billing account that funds the Gemini API
OpenAI account: I'd have to create one and bill it
Groq account: same, plus a separate API to learn

Every new third-party SaaS account is a future maintenance cost: a new dashboard, a new key to rotate, a new bill to forget about until the credit card statement arrives. If the Gemini key I already had could do voice, that was a strong default.

It can. Gemini 2.5 Flash accepts audio inputs natively as multimodal parts. Same generateContent endpoint that my existing AI features use, just with an inline_data part that carries the audio bytes.

Why not Web Speech API (the "free" option)?

The browser actually ships a SpeechRecognition API. It's free. So why not use that?

Three reasons that killed it for my use case:

Browser support: Chrome and Edge only. Firefox doesn't implement it. Safari has partial support. For a journaling app where users are likely on whatever browser they prefer, that's a non-starter.
Privacy: Chrome's implementation routes audio to Google's servers. My app's whole pitch is "your honest journal, not a feed for ad targeting." I'm not handing the most intimate audio my users produce to an ad company by default.
Control: I can't tune the model, can't set retention, can't see what was sent. With my own backend in front of Gemini, I control all three.

Native mobile is different. expo-speech-recognition runs on-device using iOS Speech and Android SpeechRecognizer. That's free and local and good. So my architecture is hybrid: native STT on mobile, Gemini on web.

The edge function

Here's the meaningful core. Full file is ~200 lines; I've stripped boilerplate.

// supabase/functions/transcribe-audio/index.ts
import { createClient } from 'https://esm.sh/@supabase/supabase-js@2.101.1';

const GEMINI_API_KEY = Deno.env.get('GEMINI_API_KEY') || '';
const SUPABASE_URL = Deno.env.get('SUPABASE_URL') || '';
const SUPABASE_SERVICE_ROLE_KEY = Deno.env.get('SUPABASE_SERVICE_ROLE_KEY') || '';

const GEMINI_MODEL = 'gemini-2.5-flash';
const GEMINI_URL = `https://generativelanguage.googleapis.com/v1beta/models/${GEMINI_MODEL}:generateContent`;

// Inline audio parts up to 20 MB; ~25+ min of opus speech.
const MAX_AUDIO_BYTES = 20 * 1024 * 1024;

Deno.serve(async (req: Request) => {
  if (req.method !== 'POST') return jsonResponse({ error: 'Method not allowed' }, 405);

  // 1. Verify the user JWT ourselves (config.toml has verify_jwt = false because
  //    we need to parse multipart audio and read the Authorization header in
  //    one place; the platform's pre-check would conflict).
  const authHeader = req.headers.get('Authorization');
  if (!authHeader?.startsWith('Bearer ')) return jsonResponse({ error: 'Unauthorized' }, 401);

  const supabase = createClient(SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY, {
    auth: { persistSession: false, autoRefreshToken: false },
  });
  const { data: { user }, error } = await supabase.auth.getUser(
    authHeader.slice('Bearer '.length),
  );
  if (error || !user) return jsonResponse({ error: 'Unauthorized' }, 401);

  // 2. Parse the upload
  const formData = await req.formData();
  const audio = formData.get('audio');
  if (!(audio instanceof Blob) || audio.size === 0) {
    return jsonResponse({ error: 'Missing audio' }, 400);
  }
  if (audio.size > MAX_AUDIO_BYTES) {
    return jsonResponse({ error: 'Audio too large' }, 413);
  }

  // 3. ... rate-limit check goes here, see below ...

  // 4. Encode for Gemini inline
  const base64Audio = bufferToBase64(new Uint8Array(await audio.arrayBuffer()));
  const mimeType = (audio.type || 'audio/webm').split(';')[0].trim().toLowerCase();

  // 5. Call Gemini
  const body = {
    contents: [{
      role: 'user',
      parts: [
        { text: 'Transcribe this audio. Return ONLY the words spoken, with normal punctuation.' },
        { inline_data: { mime_type: mimeType, data: base64Audio } },
      ],
    }],
    generationConfig: { temperature: 0.2, maxOutputTokens: 8192 },
  };

  const res = await fetch(`${GEMINI_URL}?key=${encodeURIComponent(GEMINI_API_KEY)}`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(body),
  });

  if (!res.ok) return jsonResponse({ error: `Gemini ${res.status}` }, 502);

  const parsed = await res.json();
  const transcript = (parsed.candidates?.[0]?.content?.parts?.[0]?.text ?? '').trim();
  if (!transcript) return jsonResponse({ error: 'Empty transcript' }, 502);

  // 6. ... usage logging goes here ...

  return jsonResponse({ transcript, source: 'gemini_flash' });
});

// btoa() blows the stack on large arrays, build the binary string in chunks
function bufferToBase64(buf: Uint8Array): string {
  const CHUNK = 0x8000;
  const parts: string[] = [];
  for (let i = 0; i < buf.length; i += CHUNK) {
    parts.push(String.fromCharCode(...buf.subarray(i, i + CHUNK)));
  }
  return btoa(parts.join(''));
}

function jsonResponse(body: unknown, status = 200): Response {
  return new Response(JSON.stringify(body), {
    status,
    headers: { 'Content-Type': 'application/json' },
  });
}

Three small things worth flagging:

The Authorization parse. If you let Supabase's edge runtime verify the JWT for you, it adds a step before your function runs. With multipart/form-data and a custom auth check we want side-by-side, it's cleaner to set verify_jwt = false in config.toml and do it ourselves. One less moving piece.

bufferToBase64 chunking. I learned the hard way that btoa(String.fromCharCode(...largeArray)) will throw a stack overflow on a 5 MB buffer. The chunked variant above handles inputs up to the API's 20 MB cap fine.

The prompt. You'd be surprised how often the model wants to add commentary like "Here's the transcript:". Insisting on "ONLY the words spoken" with normal punctuation cuts this down to near-zero. Setting temperature: 0.2 further pins it.

Per-user rate limit (so one bad-faith account can't burn your credits)

The version of the function above will happily transcribe any audio you throw at it, indefinitely, until your Gemini bill goes vertical. For a public app you need a per-user budget.

I added a tiny transcription_usage table:

CREATE TABLE transcription_usage (
  id BIGSERIAL PRIMARY KEY,
  user_id UUID NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
  bytes INT NOT NULL DEFAULT 0,
  source TEXT NOT NULL DEFAULT 'gemini_flash',
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE INDEX idx_transcription_usage_user_time
  ON transcription_usage (user_id, created_at DESC);

ALTER TABLE transcription_usage ENABLE ROW LEVEL SECURITY;

CREATE POLICY "users read own transcription usage"
  ON transcription_usage FOR SELECT USING (auth.uid() = user_id);
-- No INSERT policy: only the service role (the edge function) writes.

Then in the function, between the auth/parse step and the Gemini call:

const DAILY_REQUEST_LIMIT = 50;
const DAILY_BYTES_LIMIT = 100 * 1024 * 1024; // 100 MB
const ROLLING_WINDOW_HOURS = 24;

const sinceIso = new Date(
  Date.now() - ROLLING_WINDOW_HOURS * 60 * 60 * 1000,
).toISOString();

const { data: usageRows, error: usageErr } = await supabase
  .from('transcription_usage')
  .select('bytes')
  .eq('user_id', user.id)
  .gte('created_at', sinceIso);

if (usageErr) {
  // Fail-closed: better to block one request than risk uncapped spend.
  return jsonResponse({ error: 'Usage check failed' }, 503);
}

const requestCount = usageRows?.length ?? 0;
const bytesUsed = usageRows.reduce((sum, r) => sum + (r.bytes ?? 0), 0);

if (requestCount >= DAILY_REQUEST_LIMIT) {
  return jsonResponse({
    error: 'Daily transcription limit reached',
    limit: DAILY_REQUEST_LIMIT,
    used: requestCount,
    window_hours: ROLLING_WINDOW_HOURS,
  }, 429);
}
if (bytesUsed + audio.size > DAILY_BYTES_LIMIT) {
  return jsonResponse({ error: 'Daily transcription size budget reached' }, 429);
}

And after a successful Gemini call:

await supabase.from('transcription_usage').insert({
  user_id: user.id,
  bytes: audio.size,
  source: 'gemini_flash',
});

A few things I learned about this pattern:

Fail closed on the usage check. If the table query fails, return 503 and refuse to call Gemini. The opposite (call Gemini anyway, log nothing) opens an uncapped-spend hole if your DB has a bad day.
Bytes is a better budget unit than seconds. I don't trust my client's audio metadata, but I always know how many bytes I uploaded.
Don't put the limit number in the user-facing error. I do it above for code clarity, but production should say "Daily transcription limit reached. Try again later." A bad-faith user gets less information for crafting a bypass.
Belt and braces with a GCP budget alert. Set a $5/month cap on the Gemini project at console.cloud.google.com/billing, alerts at 50/90/100%. If my edge-function rate limit ever has a bug, the budget catches it.

For my own usage (~5 voice notes/day at 30 seconds each) the actual Gemini bill is about $0.06/month. The 50/100 MB cap is generous-honest, tight-enough-to-not-bleed.

Browser side

The web client is a thin wrapper around MediaRecorder:

// pickAudioMimeType() picks the first one MediaRecorder.isTypeSupported() returns true for:
//   audio/webm;codecs=opus -> Chrome/Edge/Firefox
//   audio/mp4;codecs=mp4a.40.2 -> Safari
//   ...
const recorder = new MediaRecorder(stream, { mimeType: pickAudioMimeType() });

recorder.ondataavailable = (e) => {
  if (e.data && e.data.size > 0) chunks.push(e.data);
};

recorder.onstop = async () => {
  const blob = new Blob(chunks, { type: mimeType });
  const form = new FormData();
  form.set('audio', blob, 'audio.webm');

  const res = await fetch(`${SUPABASE_URL}/functions/v1/transcribe-audio`, {
    method: 'POST',
    headers: { Authorization: `Bearer ${session.access_token}` },
    body: form,
  });
  const { transcript } = await res.json();
  // ... do something with transcript ...
};

recorder.start();

Two real-world tweaks I had to make:

Mime sniffing per browser. Chrome emits audio/webm;codecs=opus, Safari emits audio/mp4;codecs=mp4a.40.2. Probe with MediaRecorder.isTypeSupported() and pick the first one that passes. Pass the same mime to Gemini's inline_data.mime_type.
Permission fail messages. getUserMedia rejects with NotAllowedError when the user denied mic access, SecurityError on insecure context. Handle them with friendlier copy than "DOMException: blah blah".

What I keep on disk vs throw away

This is a journaling app, so privacy matters more than typical voice apps. My defaults:

Audio: never persisted server-side. The blob is uploaded for one-shot transcription, sent to Gemini, then dropped. Nothing in Supabase Storage by default.
Raw transcript: persisted in journal_entries.metadata.raw_transcript JSONB column, alongside the (possibly AI-organised) content. This way the user can always toggle back to "what they actually said" if the AI cleaned the wrong thing.
Provenance flag: metadata.transcription_source: 'gemini_flash' | 'native_ios' | 'native_android' | 'manual'. Useful for later analysis ("are users on the native STT path getting better quality?") and for an honest UI badge.

TL;DR cost comparison at the scale of one happy user

5 voice entries per day, 30 seconds each = 2.5 minutes per day per user.

Provider	Cost / month / user
OpenAI Whisper API	~$0.45
Deepgram Nova-3 batch	~$0.44
Groq Whisper Large v3 Turbo	~$0.06
Gemini 2.5 Flash multimodal	~$0.06 (and free under my existing GCP credits)

For a journaling app at single-user scale, none of these are scary. But the difference between $0.06 and $0.45 starts to matter when you cross 1,000 active users a month. And the Gemini path uses credits I'd already paid for elsewhere by being in the GCP free tier.

I build this kind of glue for indie products and small teams. If you're untangling a similar choice or want a similar edge function for your own stack, I'm at astraedus.dev.

I Built a Journal App That Organizes Your Thoughts With AI

Diven Rastdus — Fri, 03 Apr 2026 15:21:23 +0000

I Built a Journal App That Organizes Your Thoughts With AI

Most journal apps give you a blank page and wish you luck. You open the app, stare at the cursor, try to remember what happened today, type something half-hearted, close the app. Repeat for three days. Then stop.

The problem isn't motivation. It's design.

Why people stop journaling

After researching 13 journal apps (Day One, Notion, Obsidian, Rosebud, Reflect, Apple Journal, and more), the same four problems kept showing up:

1. The blank page. Opening an empty page with no context about your day kills the habit. You're reconstructing your entire day from memory before you even start writing.

2. Organization anxiety. "Where does this note go?" If you have to think about categories, folders, or tags before writing, you won't write.

3. No reward loop. You write entries but never see patterns, insights, or growth. It feels like shouting into a void.

4. Disconnected from your life. Your day happened across 5 apps. None of that context shows up in your journal.

The fix: dump and organize

Arc Smart Journal works differently. You dump whatever is on your mind. Text, voice, sticky notes. No title, no folder, no category. Just write.

Then AI takes over.

Every entry gets processed through a two-pass Gemini pipeline (I wrote about two-pass LLM processing in a previous article). The first pass classifies the entry independently. The second pass cross-references against your existing notes to find threads, connections, and patterns.

The output:

Auto-categorization. Your entry about "the project deadline is stressing me out, also need to buy milk" gets tagged: PROJECTS, PERSONAL LIFE.
Action item extraction. "Buy milk" and "call the dentist" become checklist items. No manual entry.
Thread detection. You wrote about the project 4 times this week. The AI groups those into an ongoing thread with a synthesis.
Connection discovery. You mentioned sleep quality on Monday and productivity on Wednesday. The AI connects them.
Pattern recognition. "You tend to write about work stress on Sundays."

Killing the blank page

The capture screen never starts empty. It pulls context from three sources:

Calendar. Using expo-calendar, the app reads your device's synced calendars (Google, Apple, Outlook, whatever you use). The placeholder shows: "You had 3 meetings today. Design review at 2pm. How's the project going?"

Location. expo-location provides your city. Saved on each note for context. "You were in Brisbane when you wrote this."

Weather. Open-Meteo API (free, no key needed) adds weather context. "22 degrees, partly cloudy." Small detail, but it anchors the memory.

Together, these solve the blank page. Instead of "What's on your mind?" you see "You had a busy morning with 3 meetings. Your calendar is clear this afternoon. How are you feeling?"

The architecture

Expo 54 (React Native)
  |
  +-- expo-router (file-based routing)
  +-- expo-calendar (device calendar access)
  +-- expo-location (city/coordinates)
  +-- expo-speech-recognition (voice capture)
  |
  +-- Supabase (auth + postgres + edge functions)
  |     |
  |     +-- journal_entries (content, mood, location, weather, theme_tags)
  |     +-- insights (threads, connections, patterns, forgotten ideas)
  |     +-- action_items (AI-extracted todos)
  |     +-- life_map_nodes (hierarchical topic tree)
  |     +-- entry_themes (note-to-topic links)
  |
  +-- Gemini 2.5 Flash (AI processing)
        |
        +-- Pass 1: classify each entry independently
        +-- Pass 2: cross-reference all entries for relationships
        +-- Action item extraction
        +-- Theme/topic assignment

The key design decision: AI processing runs asynchronously. You write and save instantly. The AI processes in the background and updates your insights. This means capture is always fast. The intelligence comes later.

Mood tracking without the burden

Five options: struggling, uncertain, steady, hopeful, alive. One tap. Optional. No sliders, no scales, no writing prompts about your feelings.

Over time, this builds a mood trend chart. A simple line showing how you've felt over the last 30 days. The patterns surprise people. "Your mood consistently dips on Sundays" or "You've been trending upward since you started exercising."

Voice dumps

The killer feature for people with ADHD (a massive underserved market for journal apps): record a 5-minute voice ramble. The raw transcript appears. Then tap "Organize" and the AI restructures it.

A scattered 5-minute voice note about groceries, work stress, and an idea you had in the shower becomes 3 clean notes with 2 action items. Your voice, your words, just structured.

The mind map

Every note you write adds to a visual map of your thoughts. Topics, sub-topics, connections. You can view it as a traditional tree or switch to a force-directed 2D graph where related topics cluster together.

This is the "second brain" that PKM tools promise but require hundreds of hours of manual linking to achieve. Arc builds it automatically from your writing. Zettelkasten results without Zettelkasten effort.

What I learned

The blank page is the real enemy. Not motivation, not features, not design. If the first thing a user sees is emptiness, they'll close the app.

AI organization beats manual organization. People don't want to build systems. They want to dump thoughts and have the system emerge.

Mood tracking works when it's one tap. The moment you add complexity (scales, journaling prompts, mandatory fields), people skip it.

Context is free value. Calendar, weather, location cost almost nothing to implement but make every entry feel richer.

Action item extraction is the dream feature. Every user I talked to said some version of "I wish my journal could create todos for me."

Stack

Expo 54 + expo-router 6
TypeScript (strict mode)
Supabase (auth, postgres, edge functions)
Gemini 2.5 Flash (AI processing)
react-native-svg (graph visualization)
react-native-chart-kit (mood trends)
expo-calendar, expo-location (context)
Open-Meteo (weather, free API)

The full app is about 30 files of TypeScript. No complex state management. No Redux. Supabase handles persistence, Gemini handles intelligence.

I build production AI systems. If you're working on AI-powered apps, I'm at astraedus.dev.

Two-Pass LLM Processing: When Single-Pass Classification Isn't Enough

Diven Rastdus — Fri, 03 Apr 2026 10:42:00 +0000

Two-Pass LLM Processing: When Single-Pass Classification Isn't Enough

Here's a pattern I keep running into: you have a batch of items (messages, tickets, documents, transactions) and you need to classify each one. The obvious approach is one LLM call per item. It works fine until it doesn't.

The failure mode is subtle. Each item gets classified correctly in isolation. But the relationships between items -- escalation patterns, contradictions, duplicate reports of the same issue -- are invisible to a single-pass classifier because it never sees the full picture.

The problem

Say you're triaging a CEO's morning messages. Three Slack messages from the same person:

9:15 AM: "API migration 60% done, no blockers"
10:30 AM: "Found an issue with payment endpoints, investigating"
11:45 AM: "3% of live payments failing, need rollback/hotfix decision within an hour"

A single-pass classifier looks at message #1 and says: "FYI, low priority." It's correct -- in isolation.

But a human reading all three messages sees an escalation from "no blockers" to "production incident requiring executive decision." The classification of message #1 should change in light of messages #2 and #3, because it's the start of a thread that ended in a crisis.

Single-pass classification can't do this. It processes each item without context from the others.

The two-pass architecture

The fix is straightforward: run the LLM twice.

Pass 1 -- Independent classification. Process each item individually. Get per-item labels: category, urgency, metadata. This is your standard classification pass. It runs fast because items can be processed in parallel.

interface Pass1Result {
  itemId: string;
  category: "urgent" | "delegate" | "fyi" | "ignore";
  urgency: "critical" | "high" | "medium" | "low";
  summary: string;
  suggestedAction: string;
}

async function pass1(items: Item[]): Promise<Pass1Result[]> {
  // Each item classified independently
  const results = await Promise.all(
    items.map(item => classifyItem(item))
  );
  return results;
}

Pass 2 -- Cross-reference and synthesis. Feed ALL items plus ALL Pass 1 classifications back into the LLM. Ask it to find relationships, patterns, and adjust classifications based on the full picture.

interface Pass2Result {
  threads: Thread[];           // Related item clusters
  flags: Flag[];               // Cross-item alerts
  adjustedClassifications: Map<string, Pass1Result>;
  briefing: string;            // Synthesized summary
}

async function pass2(
  items: Item[],
  pass1Results: Pass1Result[]
): Promise<Pass2Result> {
  const prompt = buildCrossReferencePrompt(items, pass1Results);
  return await llm.generate(prompt);
}

The key insight: Pass 2 sees what Pass 1 cannot. It catches:

Escalation threads: Items from the same source that increase in severity
Contradictions: One person says X, then reverses to Y
Scheduling conflicts: Two items reference the same time slot
Resolved issues: Problem reported, then resolved -- no action needed
Deal changes: Financial figures that shift between messages

Why not just do one big pass?

You might think: "Why not feed everything into one prompt and classify + cross-reference in a single call?"

Three reasons:

1. Classification quality degrades in long prompts. When you ask an LLM to do two things at once (classify each item AND find cross-item patterns), it tends to do both worse than if you split them. The model's attention is divided.

2. Structured output is more reliable for simple tasks. Pass 1 returns a clean, typed classification per item. No ambiguity, no free-text interpretation needed. Pass 2 can then assume correct per-item labels and focus entirely on relationships.

3. You can parallelize Pass 1. If you have 50 items, you can fire 50 parallel classification calls. Single-pass would need to fit all 50 items in one prompt, hitting context limits and increasing latency.

The prompt structure matters

For Pass 1, keep it focused:

function buildPass1Prompt(item: Item): string {
  return `Classify this message.

Channel: ${item.channel}
From: ${item.from}
Time: ${item.timestamp}
Body: ${item.body}

Respond with JSON:
{
  "category": "urgent" | "delegate" | "fyi" | "ignore",
  "urgency": "critical" | "high" | "medium" | "low",
  "summary": "one sentence",
  "suggestedAction": "what the recipient should do"
}`;
}

For Pass 2, structure the input so the model can scan efficiently:

function buildPass2Prompt(
  items: Item[],
  classifications: Pass1Result[]
): string {
  const combined = items.map((item, i) => ({
    ...item,
    classification: classifications[i],
  }));

  return `You have ${items.length} classified messages.
Review ALL messages together and identify:

1. THREADS: Groups of messages from the same person about the same topic.
   Flag if a thread escalates in severity.

2. CONTRADICTIONS: Cases where someone says X then reverses to Y.

3. SCHEDULING CONFLICTS: Multiple items referencing overlapping times.

4. RESOLVED ITEMS: Problems that were reported then resolved
   (these need no action, even if Pass 1 flagged them as urgent).

5. RECLASSIFICATIONS: Any items where the Pass 1 classification
   should change based on cross-item context.

Messages and their classifications:
${JSON.stringify(combined, null, 2)}

Respond with JSON matching this schema: { threads, flags, reclassifications, briefing }`;
}

When to use two-pass

This pattern adds latency (two sequential LLM calls instead of one). It's worth it when:

Items have relationships. Messages in a thread, tickets about the same system, transactions from the same account. If items are truly independent, single-pass is fine.
Cross-item patterns have high consequences. Missing an escalation thread or a scheduling conflict costs more than the extra 2-3 seconds of latency.
You need both per-item labels AND a synthesis. Dashboards that show individual items with classifications AND a summary view.
The item count fits in a single context window. Pass 2 needs all items at once. If you have 10,000 items, you'll need a different approach (clustering, then two-pass within clusters).

When it's overkill:

Items are truly independent (product reviews, standalone support tickets from different customers)
You only need per-item labels, not cross-item analysis
Latency is more important than accuracy

Production considerations

Caching. Pass 1 results are deterministic for a given item. If the same item appears in multiple batches, cache the Pass 1 result.

Error handling. If Pass 2 fails, you still have valid Pass 1 classifications. Degrade gracefully to single-pass results rather than failing entirely.

Cost. Pass 2 uses more tokens (all items + all classifications in one prompt). Use a fast, cheap model for Pass 1 (Gemini Flash, Haiku) and a more capable model for Pass 2 if needed. Often the same fast model works for both.

Structured output. Both Gemini (responseMimeType: "application/json") and OpenAI (response_format: { type: "json_object" }) support forcing JSON output. Use it. Parsing free-text LLM output is fragile.

const result = await model.generateContent({
  contents: [{ role: "user", parts: [{ text: prompt }] }],
  generationConfig: {
    responseMimeType: "application/json",
    temperature: 0.1, // Low temp for classification
  },
});

The general principle

Two-pass processing is a specific case of a broader pattern: separate extraction from synthesis. Pass 1 extracts structured data from each item. Pass 2 synthesizes across the extracted data.

This applies beyond text classification:

Code review: Pass 1 annotates each file, Pass 2 finds cross-file issues
Financial analysis: Pass 1 categorizes transactions, Pass 2 finds patterns
Research synthesis: Pass 1 summarizes each paper, Pass 2 identifies themes and contradictions

The architectural insight is that LLMs are better at focused tasks than multi-objective tasks. Splitting the work into two focused passes produces better results than one ambitious pass, even though it uses more compute.

I build production AI systems. If you're designing LLM pipelines, I'm at astraedus.dev.