DEV Community: Neu Software

What To Do When a Free LLM API Says 429 (A Field Playbook)

Neu Software — Thu, 15 Oct 2026 03:00:48 +0000

What To Do When a Free LLM API Says 429 (A Field Playbook)

Every free LLM API says yes right up until the moment it says no. The no arrives as an HTTP 429, usually forty requests into a demo or halfway through a batch job, and most tutorials end the story at "add a retry loop." Here is what actually works when you are building on free tiers, provider by provider and layer by layer.

First, find out which limit you hit

A 429 is not one problem, it is four. Free tiers meter you on several axes at once, and the fix depends on which one fired:

Requests per minute (RPM): you are asking too often. Smoothing helps, capacity does not.
Tokens per minute (TPM): a few long prompts ate the whole minute. Shorten contexts, cap max_tokens.
Requests per day (RPD): you are done until the reset. No code fix exists, only spreading work across days or providers.
Concurrency: two in-flight requests when the tier allows one. Queue, do not parallelize.

Read the response body and headers before you touch your code. Groq sends x-ratelimit-remaining-requests (daily) and x-ratelimit-remaining-tokens (per minute) plus retry-after, so the 429 tells you which window is empty and when to come back. The Gemini API returns 429 RESOURCE_EXHAUSTED and names the violated window in details[].quotaId, with retryDelay set for per-minute limits; its daily quotas reset at midnight Pacific. OpenRouter free-model caps jump from a small daily allowance to a larger one once the account has purchased credits, and its error text and x-ratelimit-reset-requests header say which window closed. Some providers, sadly, send nothing: if a 429 comes back bare, treat every window as closed and back off to the hour.

Then, retry like it means something

Three rules that cover most cases:

Honor retry-after when present. It is the provider telling you the truth; guessing is worse.
Exponential backoff with jitter when it is not: 1s, 2s, 5s, 10s, plus random offset, capped at a few minutes. Retrying a daily cap in a loop accomplishes nothing, so stop after 3 to 5 attempts and surface the error.
Count tokens, not requests. A retry loop tuned for RPM will still fail when TPM is the real wall, because the retry itself burns more tokens per minute.

Then, stop depending on one well

One provider's free tier is one quota pool. The structural fix is to treat free tiers as a pool of pools:

Keep several OpenAI-compatible keys (Gemini, Groq, NVIDIA NIM, OpenRouter, Cloudflare Workers AI, Cerebras and others each publish their own caps) and fail a request over to the next provider when one window closes. Because all of them speak the OpenAI chat format, your client code does not change, only the base URL and key.
Route by limit shape, not just availability. A nightly summarization job wants RPD headroom (Groq's daily allowances, or Gemini's daily caps). An interactive prototype wants RPM smoothness. A one-shot extraction can burn a per-token-limited tier without hurting anything.
Watch the terms. Several free tiers (Cohere's trial key, Mistral's La Plateforme free tier) are explicitly for evaluation, not production. OpenRouter's terms prohibit reselling API access. Read them before you build a product on top.

Hand-rolling this means a provider abstraction, a per-provider window tracker, a queue, and health checks for every one of them, which is most of a side project by itself. If you would rather not own that layer, there are routers that do it: I build FreeLLMAPI, an open-source, self-hosted router that keeps your provider keys, handles 429s and retries with backoff, falls back to other providers and models, and exposes one OpenAI-compatible endpoint at the end. The same job can be done by hand with the playbook above; the router just means you write it once and self-host it.

And log the shape of your traffic

Whatever you choose, log per-request: provider, model, latency, status, and the rate-limit headers. A week of logs tells you which window you actually bump, and that single fact decides whether you need smoothing, shorter prompts, a second provider, or honestly, the $5 paid tier. Most "we need more quota" problems turn out to be "one long prompt repeated in a loop" problems. The 429 is information. Read it before you reroll the key.

The AI agent demo worked. Here is what breaks between week one and week three.

Neu Software — Wed, 14 Oct 2026 02:28:38 +0000

Every AI agent demo works. You wire up a model, point it at a handful of documents, and it answers questions in a way that makes the room nod. Then it goes near real work: real enquiries, real payments, real customers, and it starts failing in ways the demo never showed. I build AI agents for businesses through neu Intelligence, and the gap between demo and dependable is where almost all of the actual engineering lives. This is the checklist I wish more teams ran before they call a prototype a product.

1. The data layer, not the model, decides the answer quality

Demos get a curated folder of perfect PDFs. Production gets the CRM with duplicate contacts, the spreadsheet two people edit at once, and the WhatsApp thread where the real booking details live. Before touching prompts, map every system the agent must read from and write to, and decide which one is authoritative for each fact. If two systems disagree and the agent has no rule to pick, you have built a confident rumor machine.

2. Give it a job description, not a feature toggle

An agent should have a written contract: the tasks it owns end to end, the systems it may touch, the actions it may take alone, and the ones that always wait for a person. "Handle parent enquiries" is a feature. "Draft replies to enquiry messages; never quote prices outside this list; hand anything about refunds to a human" is an operable role. Most agent failures described as hallucinations are really scope failures: nobody wrote down what was allowed.

3. Ship in shadow mode first

The safest first release is one where the agent does everything except send. It drafts, a human approves, and you log both. Two or three weeks of that gives you a measured error rate on real inputs, which is the only honest basis for widening permissions. Least privilege stops being a guess and becomes a dial you turn against evidence.

4. The escalation path is the product

Sooner or later the agent meets something it has never seen. What happens then is not an edge case, it is the core design decision: does it freeze, guess, or hand off cleanly with the full context attached? Design the hand-back first, then the autonomy. An agent with a good escalation path is production ready long before a fully autonomous one.

5. Watch the cost curve, not the demo bill

One pilot query costs nothing. The interesting number is cost per completed task as volume grows: retries, long contexts, and chatty tool loops scale badly if nobody designed for them. Measure per task from week one, and treat cost reduction as a standing feature rather than an emergency project in month four.

6. Ownership decides whether you can fix any of this

When the agent misbehaves at 9pm on a Sunday, can someone read its logs, edit its prompts, and change its tools without a vendor's permission? Code, prompts, schemas and integration details should live in systems you control, with a runbook that ships with the build. Ask any agency what you keep when the contract ends; the answer tells you what you are actually buying.

None of this is exotic. It is the same discipline operations engineers have applied to services for decades: clear boundaries, observability, gradual rollout, an owner. The models are new; the reliability playbook is not. If you are evaluating an agent build: I build these at https://neu.ie (disclosure), and the assess, prototype, build, run sequence we use is written up at https://neu.ie/process/. The short version: diagnose the real operation before quoting anything, and let week one produce a working session on your actual workflow, not a requirements document.

Your AI research folder is full of opinions that never met each other

Neu Software — Tue, 13 Oct 2026 01:53:59 +0000

Ask a hard question in five chat windows and you'll end up with five essays. Four of them will say "it depends," in slightly different fonts. It feels like research. It's filing.

The structural problem is simple: independent answers never meet. Nothing in the workflow forces the Gemini answer to confront the Claude answer. You become the courier between them, pasting summaries into each other, and the "discussion" runs at the speed of copy-paste. Worse, long parallel answers reward skimming: most people anchor on the first answer they finish reading and call the rest of it diligence.

The format change: put the models in the same room

You don't need a new model. You need a room where each participant hears the whole conversation before their turn, and where you can actually interrupt.

Live multi-party voice calls with AI participants make that possible: a small cast (two to eight) generated around the decision you type in, each with a role and a disposition, talking out loud, reacting to each other instead of to a fresh prompt. If you want to see the idea in text form, Andrej Karpathy's "LLM Council" write-up describes a judge-and-council pattern; voice just removes the courier. (We build one such app, AI Group Call, disclosure up front. The chairing advice below works with any multi-participant setup, including a group chat and a human referee: you.)

How to chair a room of AIs

Frame a decision, not a topic. "Should we add a caching layer before launch: yes or no, one reason each" produces arguments. "Thoughts on caching?" produces textbooks. The room is only as sharp as the question's edges.

Collect opening positions before discussion. Ask every seat to commit in one line. Panels without stated positions drift into polite agreement within three turns; the first speaker anchors everyone.

Name the disagreements. "What is wrong with the second answer?" gets much harder critique than "any objections?" People are politer than asked; so are models, unless the question forces the conflict.

Keep a skeptic in the room. If your setup doesn't seed one by default (ours does for rooms of four or more), assign one explicitly: "For your next two turns, your job is to argue the strongest case against where this conversation is heading." Watch whether the recommendation survives it. Most weak consensus dies right there, which is the point.

Bring receipts. Paste the error message, the churn numbers, the screenshot. A room argues far better with real constraints in front of it than with your summary of them.

Close with a vote and keep the transcript. One line each: what would change your mind, then what you'd do. The transcript is the actual deliverable. A month later you can check which voice was right, and calibrate who to trust for which kind of call.

What the format is not

It is not a benchmark, not a legal board, not a way to get five independent essays faster. Spoken turns are short by nature: good for trade-offs, follow-ups, and "why," bad for long proofs and code review. If the task is "read this spec and find the bug," go back to the chat window. If the task is "I have two options and a deadline," a room that argues out loud gets you to a decision in twenty minutes instead of an inbox of essays you'll never re-read.

The instruments exist now. Most people are still using filing cabinets.

Your AI Agent Shipped the Site. Now It Needs a Name: DNS for Agents, Headless

Neu Software — Sat, 10 Oct 2026 23:18:12 +0000

Your AI agent shipped the site. Now it needs a name: a practical guide to DNS for agents

Every guide about coding agents ends the same way: the agent writes the code, the tests pass, the preview runs on localhost:3000. Then you hit the part nobody scripts. The site needs a real hostname, and a hostname means DNS.

Deploying is automated. Naming is not. Here is how to close that gap, from cheapest to most permanent.

The gap: hosts speak HTTP, registries speak DNS

A hosting platform gives your agent a URL the moment it deploys. That URL lives on the platform's domain: my-app.vercel.app, project.pages.dev, app.railway.app. It is real, it has TLS, and it works. It is also not yours. The moment you migrate hosts, every link, cookie scope, webhook allowlist and OAuth redirect URI that names that URL breaks at once.

Buying a second-level domain fixes that, but it drags in a registrar account, a payment method, an email inbox for verification, and a DNS dashboard your agent has no keys to. Most agent setups stop at the platform URL for that reason, and then the migration pain arrives later.

There are three practical ways to give an agent a hostname it controls, without a human babysitting a registrar.

Option 1: A tunnel for previews only

Tools like cloudflared and ngrok give you a public HTTPS URL to a local port with no DNS work at all. Two properties make them previews, not homes:

The URL is temporary unless you tie it to an account and a domain you already own.
The site is only alive while the tunnel process runs on your machine.

Use tunnels to show a client a demo tonight. Do not point a portfolio, an API, or a webhook at one.

Option 2: Own a domain, hand the agent the API

The durable route is a domain you bought, plus a DNS API your agent can call. The workflow:

Buy the domain once, human hands, registrar of your choice.
Move DNS to a provider with a real API (Cloudflare, Hetzner DNS, and similar all qualify).
Give the agent a scoped API token that can only write records for that zone.
In your agent instructions, spell out the record vocabulary:
- A for an IPv4 address (a VPS, a home server)
- CNAME for a target hostname (most PaaS hosts: Vercel, Netlify, Cloudflare Pages)
- TXT for verification and for ACME DNS-01 challenges
- MX only if you also set up mail

Then the agent's deploy loop becomes fully headless: deploy, read the host's expected record, write the record, poll until the name resolves, request the certificate. Two guardrails make this safe. First, scope the token to one zone, never your whole account. Second, make the agent verify from outside: curl -I the new hostname and check the status code and the certificate before calling the deploy done. A record that exists but points at a stale IP looks identical to a failed deploy from inside the sandbox.

The cost is the domain itself plus the setup afternoon. For a main product, do this.

Option 3: Claim a hostname with one command, no registrar

If you just need a name that is yours to manage (not yourbrand.com, but a real resolvable hostname with records you control), there is a third shape: services that hand out subdomains programmatically.

I build one of these, AgentDomains (I make AgentDomains, so weigh the recommendation accordingly; the category itself is bigger than my tool). An agent claims something.makes.fyi with a single CLI command, then manages A, AAAA, CNAME and TXT records, URL forwarding, reverse proxying with HTTPS on the edge certificate, or full nameserver delegation, from the same CLI or an HTTP API. There is also an MCP server for agent clients that speak Model Context Protocol. Sign-up is instant and needs no email form; free covers 10 names, and confirmation-by-email keeps abandoned names from being pinned forever. It is not a registrar and sells no second-level domains. The point of the category: a hostname the agent controls through an API, in seconds, for zero money.

Other options in the same shape include free subdomain registries that require a GitHub login, and free dynamic DNS services that are really just record-updaters with extra steps. Pick by what your agent can actually do headless: if claiming a name needs an OAuth dance in a browser, that is a human step wearing a CLI costume.

What to check before you trust any hostname

Whatever route you take, make the agent prove the name works before it reports success:

dig +short <name> returns the record you intended
curl -Iv https://<name> completes the TLS handshake with a valid cert
the app responds with the expected status code, not a platform 404 page (a CNAME that resolves but the host never added the domain returns exactly that)
if the name fronts an API, one authenticated call succeeds end to end

Naming is the last human-shaped bottleneck in agent-built projects. A tunnel solves tonight, an API-driven domain you own solves forever, and an agent-native hostname service solves the ten in-between sites that are not worth a registrar login each. Pick per project, and make the verification loop part of the deploy script either way.

What it takes to make an AI give a straight answer about a sports game

Neu Software — Sat, 10 Oct 2026 16:32:08 +0000

Anyone who has asked a general chatbot about tonight's game knows the failure mode: it confidently cites a roster from three seasons ago, invents an injury report, and never saw a live price in its life. Language models are good at reasoning about sport. They are terrible at knowing what happened ten minutes ago. The fix is not a better prompt. It is architecture.

Here is what I learned building Betaware AI, an iOS and Android app that researches a matchup and answers with a confidence score. (I make Betaware AI. This is about how it is built, not a sales pitch.)

Separate the three jobs

The single most useful decision was splitting the pipeline into three stages that never swap roles.

Retrieval. Fetch live facts before the model reasons at all: injury reports, lineup news, recent form, head to head, and the current market prices. The model should not be trusted to know what is true today.
Modelling. Run actual statistical models over the retrieved data, not vibes. Team-level features, form curves, home and away splits. The model outputs a probability, not a paragraph.
Narration. Only now let the language model explain the result in plain English, grounded strictly in what stages one and two returned.

If you let the language model free associate across all three jobs, you get fluent nonsense. If you keep the boundaries, you get something a user can actually audit.

Make the market part of the input

A model's probability in a vacuum is trivia. The interesting question is always: what does the market think, and where is the gap? So the live line is a first class input, not a footnote. Pulling the moneyline, spread and totals from multiple venues (we read DraftKings and the Kalshi exchange, plus bet365 on some European soccer leagues), stripping the vig to get a consensus probability, and comparing that to the model's number turns "who wins?" into "what does my estimate imply versus the price?". It also means the answer is falsifiable the moment the game ends.

Ship the uncertainty, do not hide it

Every prediction carries a confidence score, and this is the part users push back on before they trust it. The answer: grade yourself in public. Past predictions in the app are checked against final results, so the confidence number is something you can hold the system to rather than a decoration. If you build anything predictive and only surface the hits, you have built a marketing tool, not a research tool.

Guardrails are a feature, not legal boilerplate

Because the subject is money, we pinned hard constraints early: 18 and over only, informational and entertainment purposes only, no wagers placed or accepted anywhere in the product, and a responsible gambling page with real guidance (budgets, time limits, never chasing losses). Predictions are analysis and can be wrong. Saying so plainly turned out to build trust rather than kill it.

What I would tell another builder

Never let the LLM fetch its own facts. Give it facts.
Model first, narrate second. The paragraph should be a rendering of the number, not the source of it.
Compare against a market or a benchmark, otherwise you cannot evaluate anything.
Show confidence, then grade it publicly. Calibration is the whole product.
Keep the answer short. Research deep, answer in one screen.

Betaware AI is live on the App Store and Google Play if you want to poke at the output side of this pipeline: https://betaware.app

What To Do When a Free LLM API Says 429

Neu Software — Fri, 09 Oct 2026 22:54:54 +0000

First, find out which limit you hit

A 429 is not one problem, it is four. Free tiers meter you on several axes at once, and the fix depends on which one fired:

Requests per minute (RPM): you are asking too often. Smoothing helps, capacity does not.
Tokens per minute (TPM): a few long prompts ate the whole minute. Shorten contexts, cap max_tokens.
Requests per day (RPD): you are done until the reset. No code fix exists, only spreading work across days or providers.
Concurrency: two in-flight requests when the tier allows one. Queue, do not parallelize.

Then, retry like it means something

Three rules that cover most cases:

Honor retry-after when present. It is the provider telling you the truth; guessing is worse.
Exponential backoff with jitter when it is not: 1s, 2s, 5s, 10s, plus random offset, capped at a few minutes. Retrying a daily cap in a loop accomplishes nothing, so stop after 3 to 5 attempts and surface the error.
Count tokens, not requests. A retry loop tuned for RPM will still fail when TPM is the real wall, because the retry itself burns more tokens per minute.

Then, stop depending on one well

One provider's free tier is one quota pool. The structural fix is to treat free tiers as a pool of pools:

Keep several OpenAI-compatible keys (Gemini, Groq, NVIDIA NIM, OpenRouter, Cloudflare Workers AI, Cerebras and others each publish their own caps) and fail a request over to the next provider when one window closes. Because all of them speak the OpenAI chat format, your client code does not change, only the base URL and key.
Route by limit shape, not just availability. A nightly summarization job wants RPD headroom (Groq's daily allowances, or Gemini's daily caps). An interactive prototype wants RPM smoothness. A one-shot extraction can burn a per-token-limited tier without hurting anything.
Watch the terms. Several free tiers (Cohere's trial key, Mistral's La Plateforme free tier) are explicitly for evaluation, not production. OpenRouter's terms prohibit reselling API access. Read them before you build a product on top.

And log the shape of your traffic

Your business is invisible to ChatGPT: an AI visibility audit you can run this afternoon

Neu Software — Fri, 09 Oct 2026 16:41:49 +0000

Ask your ten most common customers what they would type into ChatGPT to find a business like yours. Then type those questions yourself, in a clean session, and write down the answer. For most small businesses the result is uncomfortable: the model recommends a competitor, or hedges, or states something about you that stopped being true two years ago.

This is not a hypothetical problem anymore. Buyers increasingly start their search inside an assistant instead of a search results page, and the assistant names two or three businesses, not ten. Being "ranked well on Google" does not automatically make you one of the three.

Here is the honest part first: there is no trick. Google's own documentation for its generative features says optimizing for them is optimizing for search, and it explicitly tells you that a few popular "AI SEO" moves, like treating an llms.txt file as a magic tag, will not do much on their own. What earns a citation from any answer engine is boring and engineering-shaped: be retrievable, be extractable, be consistent. The audit below tests all three, and it takes about an afternoon.

Step 1: Record the baseline

Open ChatGPT, Perplexity, Gemini and (if you can) Google's AI Mode in clean sessions with no history. Ask the ten questions your buyers actually ask, phrased the way they phrase them: "how do I find a reliable tutor agency in Dublin", "what should a football academy pay for fee management software", "who builds WhatsApp agents for small businesses".

For each answer write down: whether you appear at all, whether you appear accurately, and who does appear. This document is the audit. Most people skip it and start redesigning pages, which is like renovating before measuring.

Step 2: Check whether you are retrievable at all

Answer engines assemble answers by retrieving from an index. If your pages are not indexed, no clever wording will get you cited. Check the obvious things:

Are your key pages in Google's index? (site:yourdomain.com in Google, or the URL Inspection tool)
Do your robots.txt rules accidentally block AI crawlers many engines use?
Is your content rendered client-side in JavaScript that some crawlers never execute?

A surprising number of small-business sites fail this step and never get to the content discussion.

Step 3: Rewrite for extraction, not for keywords

Assistants quote chunks, not pages. A chunk that starts with a qualified, direct answer survives summarizing; a chunk that builds to a conclusion rarely does. For each important page, make the first sentence under each heading the direct answer to the question that heading is about, then justify it. This is the BLUF pattern (bottom line up front), and it helps human skimmers just as much as models.

One more thing that shows up in every study of AI citations: models quote checkable claims. Named entities, real numbers, dates, prices. "We've run payroll for 142 staff since 2024" is quotable. "We streamline payroll" is not.

Step 4: Make your facts machine-legible

Add the semantic basics: Organization and Person schema with sameAs links so the entity behind your site is not a guess, FAQ-style question-shaped headings where people actually ask questions, and an llms.txt map of your key pages (it will not hurt; it will not save you either). Keep one consistent answer to "what is this company" everywhere your name appears, because engines cross-reference.

Step 5: Fix the facts, then repeat the audit

Now go back to Step 1's list. Most gaps were not ranking gaps; they were missing or wrong facts, or pages the engines could not read. Fix those, ask for citations where they are earned (directories, local press, real reviews), and rerun the ten questions in a month.

What this is, and what it is not

This is not a growth hack. A citation inside an AI answer often satisfies the question without a click, so measure whether you appear and whether you appear right, not just traffic. And when someone tells you they can make ChatGPT rank your site, ask which of the five steps above their service actually touches.

I build this work: I build Neu Software (https://neu.so), an AI studio building custom AI agents and products, and every build ships with the machine-readable layer baked in, with the answer-engine audit as the first step. Disclosure, as promised, and the audit itself is free to run without me.

Your Self-Hosted AI Assistant Keeps Forgetting You. Here Is How We Measured a Fix.

Neu Software — Thu, 08 Oct 2026 22:28:26 +0000

Your self-hosted AI assistant keeps forgetting you. Here is how we measured a fix.

Every guide to running your own AI assistant covers the easy half: install Ollama, point a web UI at it, add an API key for a bigger model when you need one. None of them warn you about the hard half, which shows up around week three. The assistant starts forgetting things you told it in week one. Not in a dramatic way. It just re-asks your preferences, loses a project detail you corrected twice, and confidently invents an answer that used to be on a sticky note inside its own context window.

We hit this while building ScallopBot, an open-source self-hosted assistant (we work on it, and this post links to our site once at the end). Rather than guess at a fix, we tried to measure the problem and the fix against a published benchmark. Here is what we learned, in the order you will probably need it.

First, name the failure you actually have

"Bad memory" is three different bugs wearing one coat:

Storage failure. The fact was never written down anywhere. When the conversation ended or got compacted into a summary, it was gone. If your assistant only remembers what fits in the context window, this is you.
Retrieval failure. The fact was stored, but the search could not find it. You asked about "the container thing" and nothing came back for "Docker configuration" because the retrieval was pure keyword, or pure vector, and the phrasing did not line up.
Fabrication failure. Retrieval found nothing relevant, and instead of saying so, the model generated a plausible answer from weak matches. This is the worst one, because it erodes trust in the two cases where memory was working fine.

Which fix you want depends on which bug you have. Dump the raw transcripts and grep them. If the fact is in the transcript but the assistant still forgot it, you have a retrieval problem, not a storage problem.

A memory layout that survives restarts

Keep the whole memory store in one file. We use SQLite: memories, relations between memories, and session transcripts all live in one database. This sounds like a trivially boring choice and it is, which is the point. Backups are one file copy. Other processes (skills, an MCP server) can open the same store concurrently in WAL mode. You can inspect it with the sqlite3 CLI when the assistant does something strange, which beats any observability dashboard for debugging memory.

Two details matter more than the schema:

Store dates inside the memory, not beside it. "We moved the staging server last month" is useless to retrieval unless the date is embedded with the fact. Later, questions like "what changed since the review?" have something to match against.
Store what you rejected. When a newer fact contradicts an older one, keeping both and marking the old one superseded costs you a little retrieval noise. A wrong overwrite loses information permanently. Asymmetric mistakes deserve asymmetric caution.

Retrieval: two searches and a gate

Single-method retrieval is where most home-built assistants plateau. Hybrid retrieval fixes a surprising amount:

BM25 keyword search catches the exact name, ticket number, or identifier that embeddings blur away.
Dense vector search catches the paraphrase that keyword search misses entirely.
Merge the two result sets, then optionally rerank with an LLM before anything enters the context window.

The underrated piece is the gate at the end. Return results only above a relevance threshold; if nothing clears the bar, inject nothing and let the model say it does not know. That one check is the difference between an assistant with occasional gaps and an assistant that lies to you about having gaps.

Maintenance: memory needs a night shift

A memory store that only grows gets noisier to search every month. Ours runs on three clocks: a light decay tick every minute, a deeper pass roughly hourly that summarizes idle sessions and audits what has been forgotten, and a nightly consolidation cycle that merges near-duplicates, links related memories, and prunes what stopped being useful. The nightly pass is where the real cleanup happens, and it costs almost nothing because it runs while you sleep. If you build your own, start with the nightly job: dedupe on high embedding similarity, and reinforce instead of duplicating when the user repeats a fact.

Does any of this actually help? Numbers.

We ran this architecture against LoCoMo, a public benchmark of long-conversation memory (1,049 QA items across 138 sessions), using identical models and embeddings on both sides. The hybrid assistant with reranking and the score gate scored F1 0.48 against 0.38 for a file-based baseline. The more telling split: on adversarial questions with no correct answer stored, the gated system scored 0.97 against 0.77. Most of the gap is the refusal to confabulate, not smarter recall.

Two honest caveats. These numbers describe one benchmark and one model pair, not a universal league table; treat them as "this direction works" rather than "this wins." And the whole cognitive pipeline is cheap: our estimate from the per-operation cost table is about 5 to 10 cents a day of model spend at 100 messages a day, on top of a $5 to 8 monthly VPS, or nothing extra if you run it on hardware you already have.

If you want to poke at the implementation, read the memory architecture notes and the cost breakdown at scallopbot.com, or clone it and run your own numbers. The project is MIT licensed, and the memory store is one SQLite file you can open with your eyes open. That, more than any F1 score, is the actual point of self-hosting this stuff.

Stop scripting interview answers. Build a panel.

Neu Software — Thu, 08 Oct 2026 21:06:17 +0000

Most interview prep fails for the same reason: it happens in your head. You read sample answers, you nod, you feel ready — and then someone asks "tell me about a time it went wrong" and the words come out sideways. Rehearsing silently builds familiarity. Rehearsing out loud, on the clock, with follow-ups you didn't choose, builds the thing interviews actually test.

The fix isn't another list of questions. It's pressure, spoken, repeated. Here is a progression we see work, roughly one hour per session, three sessions a week.

Stage 1: Bullets, never scripts

Write each of your eight to twelve stories as bullets — situation in two lines, action in three to five, result quantified. Never full sentences. A memorised script collapses the moment somebody interrupts; bullets survive interruptions because you're re-assembling the language every time, which is exactly what a live conversation asks of you.

Test it: have someone cut you off mid-story with a tangential question. If you can answer and slide back into the narrative, the story is yours. If you lose your place, you memorised, you didn't internalise.

Stage 2: Solo reps, out loud and recorded

Time yourself answering each story cold — target 90 to 120 seconds. Record on your phone. Listen back at the start, not the whole tape: the first thirty seconds usually expose the filler words, the slow start, the result you forgot to state. This is the cheapest feedback loop in prep and the most-skipped.

Stage 3: One AI interviewer is fine. It has a blind spot.

Voice mode in any major assistant makes a serviceable mock interviewer: paste a prompt that sets a demanding persona, one question at a time, push for specifics, no breaking character. It's genuinely good at this.

But a single model tends to agree with itself. Ask it to be harsh and it often softens within three questions. Ask the same story from a different angle the next day and you get the same polite scorecard. You're rehearsing against one temperament — and the real panel in front of you will not be one temperament.

This is the gap our team decided to build for: AI Group Call puts you on a live voice call with several AI participants at once — six by default — each with a different role and disposition, and rooms of four or more include one whose job is to challenge the group's conclusions. For rehearsal that changes the texture: the skeptical CFO, the friendly recruiter, the detail-obsessed hiring manager push simultaneously, the way a real panel does. (Disclosure: we build AI Group Call. The advice above stands on its own — you can run this progression with any assistant and a stopwatch.)

Stage 4: Human mock, at least once

No tool reads energy, eye contact, or the moment you talked for six minutes without checking the room. Book one human mock — a friend, a peer, a paid session — in the final week. Ask them only for the three things AI can't see: where your energy dropped, how many times you said "um", whether you sounded defensive under pushback.

Stage 5: Taper, like a race

Stop intensive rehearsal about twelve hours before the interview. Over-rehearsed delivery reads as scripted, and the contrast when someone asks a curveball — polished to stumbling — is precisely what interviewers flag. Do one light warm-up run of your top three stories, thirty minutes before the call, purely to get your voice warm.

The whole plan is boring on purpose: bullets, reps, recordings, a panel that argues back, one human, a taper. Nobody fails this plan from lack of talent — they fail it from skipping the out-loud part. Don't skip it.

What actually changes when AI models talk to each other before answering you

Neu Software — Wed, 07 Oct 2026 20:41:47 +0000

Karpathy's llm-council showed a lot of people something quietly important: the
most useful part of asking five models a question is not reading five answers. It
is watching them review each other. His weekend project runs three stages — every
model answers, every model ranks the anonymised answers of the others, a chair
model synthesises — and people keep finding that the ranking stage surfaces things
the first stage never said.

We have been building in the same direction (AI Group Call
— disclosure: our product; it runs each seat on a model you pick from the major
labs), and the pattern holds up in a different format: a live voice call where each participant
hears the whole conversation before its turn. Three things change when models stop
answering in isolation:

Positions move. In independent answers, nothing ever changes its mind. In a
shared conversation you see it happen: model B starts certain, hears model A's
constraint it hadn't considered, and walks back its own suggestion. That retraction
is the highest-signal moment in the whole session — it tells you which consideration
actually mattered, something five parallel chats never show you because they never
had to disagree.

Errors get caught by the room, not by you. One model hallucinating an API flag
tends to get corrected by another seat, roughly the way a colleague says "that
wasn't in the docs last I checked." It is not a substitute for verification, but it
beats you being the only reviewer of five confident paragraphs.

The format forces brevity. Spoken turns are short. Counter-intuitively, that is
a feature for decisions: you get "no, because X" instead of four thousand words of
"it depends." For long-form analysis, written councils still win — a voice room is
a debate club, not a research library.

Making the argument actually produce a decision

Running the room is a skill. What we have seen work across hundreds of sessions:

Name a concrete goal, not a topic. "Should our solo app charge $9 or $19 a month?" gets a room moving. "Pricing" gets five essays.
Give seats conflicting jobs. A champion, a CFO-shaped skeptic, a customer voice. Rooms where everyone is "helpful assistant" converge on mush.
Make each seat commit before the discussion. Ask for one-line answers up front; the debate is then about why the answers differ, which is the whole point.
Say the disagreements out loud. When two seats contradict each other, ask both to respond in one sentence. That is where the actual trade-off lives.
End with the chair. Ask the most senior seat to summarise: what changed, what is still unresolved, what to do next. If you keep transcripts (ours saves them with every call), paste the summary where you work.

What this is not

Not a benchmark — confident is not correct, and a model agreeing with you is not
evidence. Not a code reviewer; spoken turns are too short for long proofs. Not a
legal or financial board. The niche where a live multi-model room genuinely beats
both single-model chat and multi-tab comparison is decisions with trade-offs:
architecture calls, pricing questions, "ship or polish" arguments, rehearsing a
pitch against investors who interrupt.

Whatever tool you reach for, the takeaway is architectural, not product-specific:
the unit of AI leverage is moving from the answer to the argument between answers.
Build your workflow around getting that argument — cross-review, ranked dissent,
or just a room where the models can hear each other — and the model you pick
matters a lot less than it feels like it should.

You Gave the Conference Talk — a Practical Workflow to Turn One Recording Into a Month of Shorts

Neu Software — Wed, 07 Oct 2026 06:22:40 +0000

You gave the talk. Slides are up, the room clapped, and the recording is sitting in a shared drive where it will stay until the storage bill arrives. Here is a workflow that turns that one recording into roughly a month of short-form posts, without hiring an editor.

Start from the transcript, not the timeline

The single biggest workflow mistake is scrubbing the video looking for good moments. Human attention anchors to visuals, so you end up picking the moments that looked good on stage, which are rarely the moments that work without the room.

Pull a transcript first (most conference organizers send one, and any decent speech-to-text tool will produce one in minutes). Read it the way a stranger would scroll a feed: which two-to-four sentences make a complete point with no context? Those are your clips. A sentence-boundary cut almost always beats a mid-sentence cut, even if the visual is slightly less exciting.

Pick a fixed clip budget before you cut

Open-ended clipping never ends. Set a budget for one talk — a realistic number is six to eight clips — and give each clip a job:

one contrarian or opinion clip (the thing you actually argued for)
one concrete tip the audience can use Monday morning
one story or failure (this is the one that travels)
two or three answers to questions you hear at every conference
one close or thesis clip

With a fixed budget and assigned jobs, clipping becomes a checklist instead of a mood.

The vertical problem on stage footage

Conference stages are wide, speakers are small, and 9:16 crops the stage away. Two options:

Static center crop — acceptable only if the speaker stands still at the podium.
Active-speaker tracking — the frame follows the person as they move.

If you do this by hand in a desktop editor, expect an afternoon per clip of keyframing. This is exactly the task automated repurposing tools have gotten good at: the ones worth using read the transcript to choose moments, cut on sentence boundaries, and track the speaker in the crop. Video To Reel is the phone app our team built for this case (disclosure: I work on it) — you pick clip count and length band, and it returns cuts with word-timed captions and source timestamps; there are desktop alternatives too, and the workflow below applies to any of them.

Captions are not optional

Plenty of short-form viewing happens muted, and conference audio is hostile: reverb, Q&A bleed, a lavalier that hisses on sibilants. Burned-in, word-timed captions are the difference between a watch and a scroll. Keep them in the middle third of the frame — the top is covered by app UI and the bottom by the caption and like rails on every major platform.

Ship on a schedule, not in a burst

Posting all eight clips the day after the talk wastes them — the platforms treat them as the same moment. A simple cadence that works:

Week 1: the story clip, then the contrarian one (3–4 days apart)
Week 2: two practical tips
Week 3: the Q&A answers
Week 4: the thesis close, framed as "what I told the room in [city]"

Tag the event (organizers reshare, and that is the highest-leverage distribution a speaker gets), and link the full recording in the first comment or your bio, not burned over the footage.

The honest cost model

Hand-editing conference footage to vertical eats real hours once you include reframing and captioning — which is why most speakers skip it entirely. Automation moves the cost to minutes of review. What automation still cannot do is know which of your points is actually the interesting one; that judgment stays with you, and it belongs at the transcript stage, before any cutting happens.

One recording, six to eight deliberate clips, a month of cadence, zero extra meetings. That is the whole trick.

You uploaded your unpublished episode. What happens to it now?

Neu Software — Wed, 07 Oct 2026 00:50:36 +0000

You uploaded your unpublished episode. What happens to it now?

Every clip tool will take a file that has not aired yet: the interview still under embargo, the client call being cut into course teasers, the founder interview you have not cleared with the guest. Almost none of them will tell you, in one place, what happens to that file after the upload bar completes.

We build a mobile app that turns one long video into vertical clips (Disclosure: we work on Video To Reel, and the honest answers below are the ones we wrote down for our own product). The questions below are the ones we think every video-processing service should be able to answer in a single page, and what a good answer looks like.

Where does the file physically live?

"Cloud" is not an answer. Storage has an address. Ours is object storage in the EU region with Hetzner hosting the render server in Germany. If a service cannot say which country processes your uploads, assume the least favourable answer, especially if you handle embargoed interviews or client material under NDA.

How long does the source file survive?

There are three honest patterns: deleted when the job finishes; deleted on a scheduled sweep; or kept indefinitely "to improve the service." The difference between the first and the third is enormous for anyone whose footage is pre-publication. In our pipeline, deletion of the source and every intermediate segment is part of the job itself, not a nightly cron; abandoned uploads get swept within 24 hours. "Deleted after 30 days" is a retention policy, not a privacy policy.

What exactly is your transcript used for?

Modern clipping means the video becomes text: an LLM reads the transcript to choose which moments to cut. That means your unpublished words go somewhere. Ask what else that text is used for. Ours sends transcript text and timings only - no video, no audio, no account identifiers - to one model call per job, and nothing is used to train models. "We may use content to improve our services" is the clause that means yes, training.

What is in the terms about rights?

Some terms quietly claim a licence to your content "to operate and promote the service." For unaired material that is a real risk, not a boilerplate worry. A processing service should claim a licence limited to producing the output you asked for, and nothing else.

Who can see it while the job runs?

Distinguish "stored on a server the vendor controls" from "streamed through a third party with its own policy" from "viewable by support staff." Our own policy states plainly that we do not watch uploads and never share footage with anyone beyond the named processors in the policy. Ask any vendor which of the three they are, and whether they name those processors at all.

Why bother asking at all?

Because the cheapest way to leak an embargoed interview is to hand it to a tool nobody vetted. The clip itself is public eventually - the exposure is the pre-publication window, when the file is most sensitive and the vendor has the least accountability.

A useful test before your next upload: find the one sentence in their policy about deletion of source files. If it exists, the rest of the policy is probably written with care. If you cannot find it, that is also an answer.

The privacy page we wrote for our own pipeline is at videotoreel.com/privacy/ - use it as a template for the questions to ask the next video tool you trust with unaired footage, whichever tool that turns out to be.