DEV Community: Stephan Miller

Remotely Save for Obsidian: S3 and WebDAV Sync That Costs $0 on R2

Stephan Miller — Wed, 21 Oct 2026 13:00:00 +0000

I’ve used Remotely Save to sync Obsidian to my phone and iPad since the first week I started using Obsidian. I told it to use Dropbox, it worked, and I didn’t think about it. That’s the greatest praise you can give a sync plugin.

Then I started to write this post and checked the changelog. The latest release is 0.5.25, from October 20, 2024. The last commit to the repo is from November 2024. It has 2.3 million downloads and over 200 open issues. So before we start with buckets and endpoints, just know that you’re putting a two year frozen plugin on your system. I covered the plugin briefly in my guide to syncing an Obsidian vault across devices. This is the long version of the part that guide skipped: pointing it at storage you own.

Is Remotely Save Dead?
Why Bring Your Own Bucket
What It Actually Costs
- The AWS bill is mostly requests, not storage
Setting Up Cloudflare R2
Setting Up AWS S3 Without Handing Over Your Account
The CORS Step You Can Skip
WebDAV: Nextcloud, InfiniCLOUD, and Your NAS
Encryption: RClone Crypt or OpenSSL
Free vs. PRO, If You’re Picking a Backend Anyway
Settings That Bite
Should You Use It?

Is Remotely Save Dead?

I don’t know, and neither does anybody else outside the developer’s head. This much is true as of this writing.

The repo is not archived. No “this is dead now” posts. It installs from the community directory, and it still syncs my vault daily. Obsidian’s plugin API is not the type to break stuff often. So a 2024 plugin works… well… Mostly.

What worries me more is not the code. It’s remotelysave.com that handles PRO tier. “FREE during beta until Jan 1, 2027”. “The price will be set later.” That’s 10 weeks from when this posts.

Unreleased for 2 years plugin, service with a deadline, but no price. Not exactly confidence-inspiring.

I wrote it anyway because it all runs on free tier below. S3, Cloudflare R2, Backblaze B2, MinIO, WebDAV. No account needed at remotelysave.com.

If PRO service disappeared tomorrow, you’re still good. The plugin communicates directly to your bucket. There is no intermediary server in the mix. If you’re going to use a zombie plugin, this is how you should do it.

Why Bring Your Own Bucket

Obsidian Sync is the easy answer and it’s good. But the Standard plan gives you 1GB of storage.

I measured the part of my writing vault that Remotely Save syncs: 9,371 files, 1.62GB. Only 80MB of that is markdown. The rest is images, PDFs, and the junk from years of web clipping. So Standard is already too small for me, and I’d be on Plus.

The BYO storage pitch is:

The storage is yours. Your notes live in a bucket you own, in a format you can read without Obsidian.
It’s cheap or free. For a normal vault, the bill rounds to zero. I’ll show numbers in a bit.
It works everywhere Obsidian does, including iOS, where folder sync is a nightmare. I covered why in the iPhone and iPad sync guide.

Trade off is someone else’s problem becomes your problem. Obsidian Sync has a support team. You have your R2 bucket.

What It Actually Costs

I went through my own vault: 1.62GB, synced from 3 devices.

Backend	Free allowance	After that	My 1.62GB vault
Cloudflare R2	10GB storage, 1M Class A ops, 10M Class B ops per month	$0.015/GB-month, zero egress	$0
Backblaze B2	First 10GB free, egress free up to 3x what you store	$6.95/TB-month	$0
AWS S3	New-account credits only	about $0.023/GB-month, plus requests	well under $1 a month
MinIO, self-hosted	Whatever disk you have	Your electricity and your weekend	$0 and a weekend

R2 is the sleeper pick. No egress fees from Cloudflare. All those devices that download your full vault for the first time, that’s egress. 10 free gigabytes is a lot of markdown.

The AWS bill is mostly requests, not storage

AAWS is the weird numbers one. Storing 1.62GB in S3 Standard costs about four cents a month. No big deal.

What costs money is asking the bucket what’s in it. A sync starts by listing the remote files, and S3 bills LIST requests at the same rate as PUT, $0.005 per thousand. A listing returns at most 1,000 objects, so my vault takes about 10 requests just to look. Run auto-sync every five minutes on three devices for eight hours a day and you’re at around 86,000 list requests a month. That’s roughly 43 cents. More than ten times the storage bill, for checking whether anything changed.

Forty-three cents is still nothing. But it’s why people get surprised by S3 bills, and it’s one more reason R2’s flat free tier is the better deal for this.

Setting Up Cloudflare R2

This is the setup I’d pick if I were starting today. Remotely Save’s own R2 tutorial is short, and so is this.

Make a Cloudflare account and enable R2. It will probably want a credit card even though you’ll stay in the free tier. Annoying, but normal for cloud storage.
Create a bucket. Name it something like obsidian-vault.
Create an API token with Object Read & Write permission, scoped to just that bucket. Don’t give it access to everything on your account.
Save what it shows you: the access key, the secret key, and the endpoint URL. Cloudflare only shows the secret once.
In Obsidian , install Remotely Save (the plugin install walkthrough covers that), open its settings, and pick S3 or compatible as the remote service.
Fill in the endpoint, region us-east-1, access key, secret key, and bucket name. R2 doesn’t really use regions, and us-east-1 is what the tutorial says to put there.
Turn on Bypass CORS. The tutorial calls this “usually what you want,” and it is.
Click Check Connectivity , then run your first sync.

Back up the vault before that first sync. Copy the folder somewhere Remotely Save doesn’t know about. Every sync tool I’ve covered in this series gets the same advice, because the first sync is when you find out you configured something backwards.

The one R2 issue: older versions of the plugin failed the connectivity check even when the settings were right, because Cloudflare doesn’t allow that kind of key to run HeadBucket. Version 0.3.29 switched to ListObjects and fixed it. Since the current version is 0.5.25, this only bites you if you’re running something ancient. If the check fails, look at your keys before you blame Cloudflare.

Setting Up AWS S3 Without Handing Over Your Account

R2 asks for a scoped token and you’re done. AWS makes you build the scope yourself, and this is where it’s easy to get stuck. The easy move is to use your root account’s keys because they’re right there.

Don’t. Those keys can do anything to your entire AWS account, and now they live in a plugin settings file on your phone.

Instead, create an IAM policy that can touch one bucket and nothing else. This is the policy from the plugin’s S3 user policy doc, with my-bucket swapped for your bucket name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ObsidianObjects",
            "Effect": "Allow",
            "Action": [
                "s3:HeadObject",
                "s3:ListBucket",
                "s3:PutObject",
                "s3:CopyObject",
                "s3:UploadPart",
                "s3:UploadPartCopy",
                "s3:ListMultipartUploads",
                "s3:AbortMultipartUpload",
                "s3:CompleteMultipartUpload",
                "s3:ListObjects",
                "s3:ListObjectsV2",
                "s3:ListParts",
                "s3:GetObject",
                "s3:GetObjectAttributes",
                "s3:DeleteObject",
                "s3:DeleteObjects"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

Then:

In IAM, create a new user and choose Attach policies directly. Attach the policy above.
Open that user, go to Security credentials , and create an access key. If it asks for a use case, pick Other.
Put that access key, the secret, your bucket name, and the bucket’s region into Remotely Save.

The doc says that the bucket itself doesn’t need a policy, only the user. It also says “not fully tested,” which is good of them to admit, and a reason to test a sync with a throwaway note first.

The CORS Step You Can Skip

If you’ve read an older Remotely Save guide, you’ve seen a big block of CORS JSON with origins like app://obsidian.md and capacitor://localhost. It used to be the step that broke everybody’s setup.

You probably don’t need it anymore. The plugin’s CORS doc says it’s only required on Obsidian desktop below 0.13.25 or mobile below 1.1.1. Newer Obsidian lets the plugin make requests that don’t go through the browser’s CORS rules, which is what the Bypass CORS toggle is about. Those version numbers are years old. If you are running Obsidian that old, CORS is not your biggest problem.

So if a tutorial sends you into your bucket’s CORS settings, skip it, turn on Bypass CORS, and only come back to it if the connectivity check fails. WebDAV has its own CORS doc with the same desktop and iOS cutoffs, plus one exception: it says Android needs CORS on the WebDAV server at any version, and Nextcloud and ownCloud don’t turn it on by default. If you sync from an Android phone over WebDAV, read that doc before you skip anything.

WebDAV: Nextcloud, InfiniCLOUD, and Your NAS

WebDAV is the other free-tier backend, and it’s the one for people who already run a server or own a NAS. Remotely Save has setup docs for a dozen of them. Here’s the 3 most people will care about:

Nextcloud. If you already self-host it, this is the obvious pick. Your notes land next to everything else you run.
Synology NAS. Turn on the WebDAV Server package and point the plugin at it. It’s great at home. Away from home you’ll need a way to reach the NAS, and exposing it to the internet so your phone can sync is something to do only if you understand what you’re letting in.
InfiniCLOUD (it used to be called TeraCLOUD). A hosted WebDAV service with a free plan that recent listings put at 20GB. The catch is that free accounts can be deleted after two years without a login. A sync plugin that’s working perfectly never makes you log into the website, so put a reminder in your calendar.

Setup is the same every time: pick Webdav in the plugin, enter the server address, username, and password, and check connectivity. The plugin puts your vault in a subfolder named after it, so one WebDAV account can hold several vaults without them stepping on each other.

I’ve never run my vault through WebDAV. I’ve used Dropbox the whole time. So for this section I’m going on the plugin’s docs, not my scars. If you hit something weird with a specific server, the per-server tutorials in the repo’s docs/remote_services/ folder are where to start.

Encryption: RClone Crypt or OpenSSL

If you set a password, Remotely Save encrypts your files on your device before they go up. Leave it blank and your notes sit in the bucket as plain markdown. With your own S3 bucket, plain is defensible. With a third-party WebDAV host, set a password.

You get two formats, and the plugin recommends RClone Crypt. Its own comparison page lays out the difference:

	RClone Crypt (recommended)	OpenSSL enc
Content	XSalsa20Poly1305, in chunks	AES-256-CBC
Filenames	Each path segment encrypted separately	The whole path encrypted as one
Reading it without Obsidian	`rclone` can mount the bucket decrypted	Write your own script

The filename row is the real tradeoff. RClone Crypt encrypts each folder name and filename on its own, so your folder structure survives. Someone with access to the bucket can’t read your note titles, but they can see that you have a folder with 400 things in it, nested three deep. OpenSSL hides the structure too, at the cost of not being able to open your vault with a standard tool.

I’d still take RClone Crypt. Being able to run rclone mount and read my notes without Obsidian is the point of owning the bucket.

Two warnings from that page. First, if you ever switch formats, you have to delete the remote files “manually and fully” so the plugin can re-upload everything. Second, the page says the author “doesn’t have sufficient knowledge to ‘judge’ which one is the better format.” Credit for saying so, and a reminder that this is a hobby project’s encryption, not an audited product. Use a long password. Also, for Dropbox, OneDrive, and WebDAV, the vault’s name itself isn’t encrypted. Don’t name your vault after something you’re hiding.

Free vs. PRO, If You’re Picking a Backend Anyway

The split as of 0.5.x:

Free, no account: S3 and anything S3-compatible (R2, B2, MinIO), Dropbox, WebDAV, OneDrive’s App Folder, and Webdis, which the README calls “mostly experimental.”

PRO, needs a remotelysave.com account: Google Drive, Box, pCloud, Yandex Disk, Koofr, Azure Blob Storage, full OneDrive access, and Smart Conflict.

Smart Conflict is the PRO feature I’d actually miss. On the free tier, the plugin detects a conflict and you pick a rule: keep the newer file or keep the larger one. Whichever loses is gone. Smart Conflict tries to merge two versions of a small markdown file instead, and for anything it can’t merge, it keeps both.

The free list is exactly the backends this post is about. If you choose R2, B2, S3, or WebDAV, the PRO price that gets announced after January 1 doesn’t touch you.

Settings That Bite

Most of these come from the README, which I read properly for the first time while writing this:

Files starting with . or _ don’t sync by default. If you have an _attachments folder, it isn’t going anywhere. Your .obsidian config folder can sync, but that option is marked experimental.
Auto-sync only runs while Obsidian is open. It’s an interval timer inside the app. Close the app and nothing syncs. Open it on your phone and nothing is current until the next run, so hit sync by hand before you start writing.
Errors “fail silently.” That’s the README’s phrase. A broken auto-sync looks exactly like a working one with nothing to sync. Check the last sync time once in a while.
Big files are slow on mobile. The README flags trouble at 50MB and up. There’s a Skip Large Files setting and a regex filter for paths. Use them before your phone tries to pull down a 300MB PDF over cell data.
data.json holds your keys. It’s in .obsidian/plugins/remotely-save/, and it has your secret key or tokens in it. The plugin drops a .gitignore into its own folder that excludes it. And one I did to myself. My laptops sync the vault through the Dropbox app, and my mobile devices sync it through Remotely Save. That plugin folder sits inside the vault, so Dropbox syncs each laptop’s copy of the plugin’s settings file. When I went looking just now, I found this in it:

data (Stephan Miller's conflicted copy 2025-12-14) (Stephan Miller's conflicted copy 2025-12-14).json

A conflicted copy of a conflicted copy. That’s my own rule from the pillar, “don’t run two sync systems on one vault,” broken by me in my own vault, and it’s been there since December. It’s harmless here, since the plugin ignores those files. But it’s the kind of mess you get when two tools both think they’re in charge.

Should You Use It?

If you want Obsidian on every device without a subscription, and you’re fine owning a bucket, yes. With a caveat I didn’t have to write a year ago.

Remotely Save has done its job for my phone and iPad for years and it’s still doing it today. R2 makes it free for most vaults. AWS makes it cost pocket change, mostly for listing files. WebDAV is free if you already run the server. Encryption is there, with the author’s own disclaimer attached.

But it hasn’t had a release since October 2024, and nobody is telling us whether that’s a pause or the end. So set it up the way that survives the plugin going away: a free-tier backend, RClone Crypt if you encrypt, and a backup that isn’t the sync target. If the plugin stops working someday, your notes are still in a bucket you own, readable with rclone. That’s a better exit than most sync tools give you.

If you’d rather someone else worry about all this, Dropbox is the simplest Remotely Save backend, and Syncthing skips the cloud entirely. Me, I’m keeping my setup. I should probably delete that conflicted copy, too.

Gemini 4 Argon Is for Sale Now. The Price Doubles December 1.

Stephan Miller — Tue, 20 Oct 2026 13:00:00 +0000

For two issues in a row I printed a second cheapskate table, because the model at the top of the first one wasn’t for sale. Gemini 4 Argon sat at number one in every Arena category, and you couldn’t buy it unless Google had vetted you as a cyber defender for its Fairwind program.

You can buy it now. Google made it generally available on October 13, and OpenRouter listed it the same day. So I deleted the second table. Then I went to see what the first wave of paying customers had to say, and the top thread on r/GeminiAI is titled “Out of Gemini 4 Argon usage after 10 messages.” You know how that goes.

Argon wasn’t the only thing going on. DeepSeek pushed a “minor update” to its Flash model on Friday, and by Sunday it was the busiest model on OpenRouter. Meta, which went closed back in April, put out open weights again. Reflection shipped a 501B open model with a benchmark claim that the independent numbers don’t back up. StepFun’s weights showed up after all. And somebody broke into OpenCode Console and spent people’s API credits for them.

Gemini 4 Argon went on sale, and the first complaint was the limits
- What the paying customers said
DeepSeek shipped a “minor update” and took the daily top spot
- The price depends on what time it is
Open weights week
- Meta went back to open, sort of
- Reflection Beam 1.0: big claim, small index
- Step 5’s weights showed up, and the free traffic left
- Space Bunny got a name
The cheapskate table: one table again
- GLM-5.3-Flash missed Overall by six hundredths of a point
- The rest of the table
- Speed is still the catch
Local sweet spot
Horror stories
- OpenCode Console leaked people’s API keys
- The Argon usage wall
- The peak-hour batch job
What’s coming
Finally

Gemini 4 Argon went on sale, and the first complaint was the limits

As of October 13, Argon is generally available in the Gemini API on the paid plan (the model string is gemini-4-argon), in the Gemini app for AI Pro and Ultra subscribers, and in AI Studio and Antigravity for Ultra. The Fairwind-only restriction is gone for paying customers. On OpenRouter it’s $2 in and $10 out, with a flex tier at $1/$5 if you can wait.

That $2/$10 is still the intro price. It has an end date now: $4/$20 from December 1. So if you build a budget on Argon this month, build it on the December number.

So is it worth $10 a million? Arena still has it number one Overall (1524, now on 4,904 votes), plus Coding, Instruction Following, Hard Prompts, and Math. But it lost one. Claude Opus 5.5 took Creative Writing, 1519 to 1516. That’s three and a half points, which is nothing statistically, but it’s the first category Argon hasn’t swept since it showed up. And the Math lead is sitting on 273 votes, so I wouldn’t tattoo that one anywhere.

Artificial Analysis likes it less. Argon scores 53 on the Intelligence Index, tied with GPT-6 Astra and behind Opus 5.5 (58), Sonnet 5.5 (56), and Meta’s Muse Wonder Full (56). That’s the same split I’ve been writing about for weeks: Arena voters love it, the hard benchmarks say it’s frontier but not the top of the frontier.

What the paying customers said

Reddit was reachable this week, so I got real threads instead of aggregator summaries. The loudest ones are about the consumer apps, not the API:

The r/GeminiAI usage thread I mentioned up top has 733 points and 141 comments. The top comment’s theory is that Argon is just very expensive for Google to run, and the limits are how Google pays for it.
On r/AntigravityGoogle: “To all the Pro users thinking of upgrading to Ultra for Gemini 4 Argon: Don’t do it. The limits are a joke.” (430 points). Another thread there, “Gemini 4 Argon did way more damage than good” (794 points), has at least one person saying the botched release made them cancel.
The positive one is from launch week. An r/singularity thread called “Gemini 4 Argon solved hallucinations” got 1,608 points, and the top reply sums it up: “‘solved’ 15% But yeah, it’s a very nice improvement if not benchmaxxed.”

If you want Argon, I’d use the API and pay per token, where you know what you’re getting. Paying for an Ultra subscription to get Argon in the app is the move people are warning each other off of.

DeepSeek shipped a “minor update” and took the daily top spot

On October 17, DeepSeek put out DeepSeek V4.1 Flash 1017, a new snapshot of the model that was already number one on OpenRouter. It now sits behind DeepSeek’s deepseek-flash API name, and the weights went up on Hugging Face under MIT the same day.

Two days later it had the top spot on OpenRouter’s daily board. The board through October 19 has it at 3.61 trillion tokens for the day, first place, with the original V4.1 Flash second at 2.8 trillion. For the week it’s sixth at 7.34 trillion on about two days of traffic, and first on New & Trending. The original V4.1 Flash is still number one for the week (29.6 trillion), but it’s down 27 percent, so you can watch the traffic move from one to the other. DeepSeek is now 24.4 percent of every request on OpenRouter.

Artificial Analysis gives the 1017 snapshot a 45, second among the 121 open-weights models it tracks (MiMo-V2.6-Pro is first at 46). AA clocks it at 211 tokens a second, with the first token arriving in about a second. The catch is the same one I keep finding with the smart cheap models. It took 210 million output tokens to get through AA’s index, against a median of 140 million, and AA doesn’t put it on the intelligence-versus-cost frontier.

r/LocalLLaMA’s release thread has 1,117 points and 314 comments. The top comment (605 upvotes): “‘minor update’ Proceeds to obliterate GLM5.3, Kimi K3 and GPT5.6 Sol.” Another one notes that “Flash is now 700B,” which is about right going by the size of the safetensors on Hugging Face. And a 519-upvote reply warns that one of the charts going around comes from a knockoff benchmark site dressed up to look like Artificial Analysis. Check where a chart comes from before you share it.

Over on r/DeepSeek, someone ran 1017 against Claude Opus 5.5 on the same task. The top reply had the most sensible take of the week: priced like Haiku, performs closer to Sonnet than Opus. And r/opencode noticed it’s free in OpenCode (1,666 points), where the replies went straight to “you are the product” and complaints about the 200K context cap.

The price depends on what time it is

DeepSeek’s own endpoint charges $0.15 in and $0.60 out off-peak. During peak hours it’s exactly double, $0.30/$1.20. Peak is 01:00 to 04:00 and 06:00 to 10:00 UTC, Monday through Friday. That’s Beijing’s working day. In US Central time, while daylight saving lasts, that’s 8 to 11pm and 1 to 5am.

DeepSeek didn’t invent this for 1017. It’s been charging by the clock since V4.1 launched in September. But now that millions of requests a day are moving onto it, the gap costs real money. “60 cents” is the number everybody quotes and $1.20 is the number you pay if your overnight batch job runs at the wrong damn time. In my cheapskate math I use the $1.20, since that’s the rate you can’t schedule your way out of.

It’s not on Arena yet, so it doesn’t get a row in the table this week. The older V4.1 Flash is in the Overall band at 38th place.

Open weights week

Three labs put out big open-weight models in the same week, and a fourth confirmed a stealth model’s identity. None of them change the cheapskate table yet. One of them is Meta reversing itself.

Meta went back to open, sort of

Back in April, one of my first roundups led with Meta abandoning open source for Muse Spark. On October 15 Meta released Muse Spark Open, with weights on Hugging Face, SageMaker, Vertex Model Garden, and Microsoft Foundry. It’s 816 billion parameters with 42 billion active, a 1M context window, native image and video input, and it’s built for coding and agent work.

The license is the first problem. It ships under the Muse Model License, which has commercial limits, so you can download the weights and still not be allowed to do whatever you want with them. If you’re building a product on it, have somebody read that license before you write any code.

And you aren’t running this at home. The hardware floor is about 700GB of VRAM at NVFP4 (four Blackwell Ultras or eight H200s) and about 2TB at BF16. On Artificial Analysis it scores 39, behind Kimi K3 and GLM 5.3 according to Developers Digest. Meta hosts it on OpenRouter at $1.25/$4.25. Why pay $4.25 out for a 39 when DeepSeek’s snapshot scores 45 and tops out at $1.20?

I’ll still take it. Six months ago the story was Meta closing the door, and this week I could pull Muse weights off Hugging Face (if I had a rack of H200s to put them on).

Reflection Beam 1.0: big claim, small index

Reflection AI, the NVIDIA-backed startup from former DeepMind researchers, released Beam 1.0 on October 16. It’s 501B total and 23B active, under Apache 2.0, which is a real open license this time. OpenRouter has it at $0.30/$1.00.

Reflection says it scores 80.9 percent on SWE-bench Verified and is three to four times more efficient than GLM 5.2. Artificial Analysis, which averages a much broader set of hard evals, gives it a 30. It’s fast (AA clocks about 350 tokens a second), but a 30 sits a long way below what that SWE-bench number implies. This is the same vendor-table problem I flagged with Meta’s Muse Wonder last week. The lab’s chart says one thing, the independent average says another, and when they’re this far apart I go with the independent one until someone explains the gap.

Step 5’s weights showed up, and the free traffic left

Last week I said I wouldn’t count StepFun’s promised October 15 weights as shipped until there was something on Hugging Face. There is. stepfun-ai/Step-5-Preview went up on the 15th, with BF16 and NVFP4 variants, though early that morning people were reporting a 404. The license field says “other,” so read it before you use it.

I also said to watch what happened to Step 5’s traffic once the free week ended on October 16. It fell 32 percent to 8.22 trillion tokens, and it’s still fourth on OpenRouter for the week. I figured the freeloaders would bail harder than that. About a third of the traffic left, and I’d guess most of the rest is paying $2.70 out now. Arena still has it 84th Overall. In Coding it’s 63rd, missing the 50-point band by 1.3 points.

Space Bunny got a name

The free stealth model that was number one on OpenRouter three weeks ago has an identity. According to eesel, Space Bunny Alpha was MiniMax M3.1, which is what everybody guessed. MiniMax M3.1 Flash showed up on OpenRouter as a free model on October 19 and did 1.05 trillion tokens on its first day, seventh place. AA gives it 43 at about 105 tokens a second.

There are no Hugging Face weights yet, and no model card or technical report either. You can’t turn thinking off, only turn it down, and it defaults to max effort. And outside the free OpenRouter endpoint, eesel says MiniMax currently sells it through subscription plans at $20, $50, or $120 a month rather than a published per-token price. Free endpoints on OpenRouter have a habit of turning into paid ones with about a week’s notice, so I’ll check on this one next issue.

The cheapskate table: one table again

If you’re new here: for each Arena category I take the leader’s rating, subtract 50, and look for the cheapest model still above that line. The band gets computed in code from every row on the board, and I write down the first model below the line so I know nothing got cut off. The board comes from LMArena’s leaderboard dataset on Hugging Face (arena.ai’s site still blocks me), published October 15. Prices are each maker’s own endpoint on OpenRouter.

Since Argon is for sale now, there’s only one table. Argon is priced at its $10 intro rate.

Category	Leader	$ out	Cheapskate pick	$ out	Δ rating	Cheaper by
Overall	gemini-4-argon-high (1524)	$10	MiMo-V2.6-Pro (1479, #27)	$0.87	−45	11.5×
Coding	gemini-4-argon-high (1559)	$10	MiMo-V2.6-Flash* (1513, #51)	$0.28	−46	36×
Creative Writing	claude-opus-5.5-high (1519)	$20	gemini-3.7-flash-high (1491, #5)	$3.75	−28	5.3×
Instruction Following	gemini-4-argon-high (1527)	$10	gemini-3.8-flash-high (1490, #14)	$3.75	−37	2.7×
Hard Prompts	gemini-4-argon-high (1549)	$10	GLM-5.3-Flash (1500, #34)	$0.50	−49	20×
Math	gemini-4-argon-high (1529)	$10	GLM-5.3-Flash (1506, #8)	$0.50	−23	20×

*2,625 votes, 3.6 points above the cutoff. GLM-5.3-Flash sits at 1525 with 8,457 votes for 50 cents and is still the safer bet.

Band depths: Overall 40, Coding 59, Creative 17, Instruction Following 23, Hard Prompts 35, Math 38.

GLM-5.3-Flash missed Overall by six hundredths of a point

The Overall cutoff is 1473.90. The first model below it is GLM-5.3-Flash at 1473.84. That’s 0.06 points, out of a scale that runs to 1524. If it had made it, it would be the Overall pick at 50 cents and a 20x ratio. Instead the pick is MiMo-V2.6-Pro at 87 cents, for the third issue running.

And GLM pushed 11.9 trillion tokens through OpenRouter this week, second only to DeepSeek. Plenty of people clearly think it’s good enough for general work, and my table leaves it out of Overall over a gap that disappears when you round to whole numbers. I don’t round, though. 1473.84 is under 1473.90, so GLM sits this one out.

It went the other way in Hard Prompts. Last week GLM missed that band by 0.3. This week it’s in by 1.0, on 20,935 votes, and it’s the pick at 20x. It also holds Math for the third issue in a row, 8th place on that board.

The rest of the table

Opus 5.5 beat Argon in Creative Writing, so that ratio now runs against Opus’s $20 instead of Argon’s $10, and Gemini 3.7 Flash stays the pick at 5.3x. Not one open model made that 17-model band. Instruction Following looks the same: the cheapest thing in its band is Gemini 3.8 Flash at $3.75. If you write fiction or need strict instruction following, you’re still paying for quality.

MiMo-V2.6-Flash holds Coding for the third week, and it went up this time, 1,989 votes to 2,625. It’s still only 3.6 points over the line. My rule was that if it held the spot on a few thousand votes I’d call it the Coding pick for real. It’s getting there. I’d still run GLM-5.3-Flash for anything I cared about.

Mark December 1 on the calendar, because every ratio with Argon as the leader doubles that day. Overall goes from 11.5x to 23x, and Hard Prompts and Math go from 20x to 40x, without a single pick changing. Google raises its price and the cheap models start looking twice as good.

Speed is still the catch

The cheap end of the table is still slow. MiMo-V2.6-Pro, the best open model on AA at 46, runs at about 50 tokens a second. MiMo-V2.6-Flash is around 54 and GLM-5.3-Flash about 78 this week (it moves around every week, so I re-pull it). Say your agent runs 20 steps and writes 1,000 tokens at each one. At 50 tokens a second that’s almost seven minutes of waiting. At the 211 that DeepSeek’s 1017 snapshot manages, it’s about a minute and a half, which is why I want to see where it lands once Arena rates it.

The distillation accusation Anthropic made against Xiaomi in September still hangs over the MiMo family. Anthropic made that claim as a competitor, and nobody has ruled on it. If your contracts care where a model’s training data came from, factor it in.

Local sweet spot

This section picks the biggest model that fits your machine with room left for context and still runs at 15 or more tokens a second. Nothing changed this week. I added Gemma 4 26B-A4B to the candidates and it didn’t beat Qwen3.8 27B on any tier. Every big open release this week (Muse Spark Open, Beam, the DeepSeek snapshot) is far too big for a 64GB machine.

Your machine	Pick	Size	Arena	AA	Measured tok/s
Mac 16GB	qwen3.5-9b Q4_K_M	5.7 GB	unlisted	-	27 (n=4, 6-82)
Mac 32GB	qwen3.8-27b MLX 4bit	16.1 GB	1438 (#100)	34	not measured
Mac 64GB+	qwen3.8-27b MLX 8bit	29.5 GB	1438 (#100)	34	19 (n=1)
NVIDIA 12GB	qwen3.5-9b Q4_K_M	5.7 GB	unlisted	-	50 (n=1)
NVIDIA 24GB	qwen3.8-27b Q4_K_M	16.5 GB	1438 (#100)	34	41 (n=2, 40-42)

The 9B rows have no Arena rating, so they’re picked by what fits, not by a leaderboard. The Mac 32GB speed is still unmeasured. I’m not going to fill it in from a calculator site.

Horror stories

OpenCode Console leaked people’s API keys

Overnight on October 18, attackers got into OpenCode Console, an older part of OpenCode’s online platform where some users store the API keys they use with OpenAI, Anthropic, and Google. TechCrunch reports that 1.4 percent of Console users got breach notices, and people reported unauthorized charges from a few hundred dollars to more than $1,000. One user: “I saw my Open AI, Anthropic, and Google AI spend alerts all fire off before going to bed.” Some had their Google Cloud projects suspended on top of that, so they lost the money and the projects in the same night.

Anthropic and Google blocked the exposed keys. OpenAI worked with OpenCode on refunds, and OpenCode says it’s refunding the charges too. Co-founder Dax Raad drew the line clearly: “Note this is the OpenCode Console, not OpenCode itself, those keys never come to us.” So the CLI didn’t leak anything. The web console did. OpenCode blamed a “large-scale distillation attack,” and some security people said that hasn’t been proven.

If you ever pasted a provider key into OpenCode Console, rotate it at the provider today. And set spend alerts on every provider account you have. Those alerts are the only reason the guy above found out before morning.

The Argon usage wall

I covered this above, but it’s the most common way people got burned this week. You pay for AI Ultra because you want Argon. You get through a handful of messages and hit the limit. The r/AntigravityGoogle advice is blunt: don’t upgrade for it. The API charges per token at a published price. The app charges a subscription and then cuts you off at a number nobody printed.

The peak-hour batch job

You move a nightly summarization job to DeepSeek V4.1 Flash because it’s 60 cents out. You schedule it at 9pm Central because that’s when your other jobs finish. That’s inside DeepSeek’s peak window, so every token costs double, and you’re paying $1.20 for a model you budgeted at $0.60. In a Hacker News thread about V4.1 Flash, one commenter said they burned through $50 in a few days on the cheapest provider on OpenRouter, a quarter of what their Codex subscription costs. Cheap per token isn’t the same as cheap per job. Check what time your jobs actually run.

What’s coming

Argon’s API price doubles to $4/$20 on December 1, and that’s the only item in this section with a firm date.

Mistral Large 4’s open weights are still scheduled for October 31. Kimi K3.1 has a rumored October 29 date, and Grok 4.8 was supposed to land around today, according to the LLM countdown tracker. I’ve learned to treat xAI dates as tweets. Claude Fable 5.5 is at about 93 percent on Manifold for October, and Anthropic hasn’t said a word. Haiku 5.5 still doesn’t have an Arena rating, two weeks after launch. MiniMax hasn’t posted M3.1 Flash weights or a model card. DeepSeek V5 is still speculation.

Finally

Every headline this week came with a condition attached. Argon is buyable, until December 1 changes the price. DeepSeek is 60 cents, unless the clock says otherwise. Meta’s weights are open, under a license you have to read first. If you only skim the launch posts, you’ll budget off the wrong number.

So what actually changed for somebody picking a model today? Less than the headlines suggest. The top of the board got a price tag. The bottom of my table held for a third week: MiMo-V2.6-Pro, MiMo-V2.6-Flash, and GLM-5.3-Flash, all under a dollar out, and I’d still start any cost-sensitive project with GLM-5.3-Flash and see whether it falls short before paying 20 times more.

Next week I’ll see whether DeepSeek’s 1017 snapshot gets an Arena rating, and whether it lands in the band at $1.20. And I’ll keep checking whether Haiku 5.5 ever gets rated.

Firecrawl Universal Scrape Can Read Podcasts. The Terms Say You Can't Keep Them.

Stephan Miller — Thu, 15 Oct 2026 12:00:00 +0000

On October 8, Firecrawl launched Universal Scrape. The scrape endpoint I have been feeding URLs to for this whole series now answers questions about podcasts, people profiles, company financials and website traffic too. Ask it about podcasts and it gets you transcripts.

The 70-line harness from the Arena post already does the hard part: run on a schedule, save every result to SQLite, compare this week to last week, speak up when something changes. Point it at podcast mentions instead of a leaderboard and you have a brand monitor. Which podcasts talked about your product this week? Is that number going up? Who mentioned it for the first time?

Then I read the terms and had to accept them to use the feature. And the terms forbid storing the data, reusing it for a later query, keeping it past the session, and showing it to anyone who isn’t one of your own users. That rules out the harness, the comparison, my AI agents, and other things I haven’t thought of yet.

So this post is not what I originally planned. It’s about the fine print for every paid provider behind Universal Scrape, plus the version of the podcast tracker that survives it.

What Universal Scrape Adds
The Recipe I Was Going to Write
What Particle’s Terms Actually Say
I Read the Terms for Every Paid Provider
The Tracker That Survives the Terms
- The second gotcha: don’t pass a domain
- The script
- What I can show you
Read the Schedule Before You Write the Recipe

What Universal Scrape Adds

Two things, really.

Files by URL. PDFs, images, Word documents, spreadsheets and presentations, if you can link to them. You pass the URL like any web page and get markdown back. The scrape docs put PDF parsing at 1 credit per PDF page. None of that has provider terms attached.

Provider data. This is Alexandria, the data catalog Firecrawl launched a couple of weeks ago, folded into the scrape endpoint. The launch post names the headliners: Apollo, FullEnrich and Data Legion for people and company profiles, Fiscal.ai and Benzinga for financial data, and Particle for podcasts.

In the CLI, a provider call is a scrape where the URL is a provider/capability address:

firecrawl scrape particle/podcasts/mentions/timeseries \
  --options '{"company_id": "anthropic", "interval": "month"}' --json

That syntax needs a recent CLI. Mine was 1.23.3, so I updated to 1.26.3.

npm install -g firecrawl-cli@latest
firecrawl --version # 1.26.3

Same plain npm install as the setup post, so nothing touched my editor configs, and the key still comes from FIRECRAWL_API_KEY.

The Recipe I Was Going to Write

Every useful scraping job in this series turned out to be the same job. A scraper tells you what a page says right now. Change needs history, history needs storage, and storage is why the harness exists.

For podcasts, the plan was one weekly call, a row per episode in SQLite, and three alerts:

mentions this week versus the last four
shows mentioning the term for the first time
the transcript line where each mention happens, so you can read it without listening to a 90-minute episode

Particle has the tools for all of it. Episode search, transcripts, mention lookups, speaker lists, even which sponsors read ads on which shows. 105 tools, at 5, 15, 25 or 50 credits a call.

What Particle’s Terms Actually Say

Before the first call, Particle makes you accept its terms from the CLI:

firecrawl alexandria terms show particle --json
firecrawl alexandria terms accept particle \
  --terms-version B-1.0.0 --digest <sha256 from the show output> --confirm

Credit to Firecrawl here. You accept an exact version, pinned by a hash of the document, and the JSON that terms show returns tells an AI agent to present the terms to a human and wait for an explicit yes. My agent did exactly that. It read them, came back, and told me the recipe was dead before it accepted anything.

The document is Schedule B, which pulls in Particle’s Data Use Terms. The parts that matter, quoted:

(g) Download, save, or store Clips, Full Episode Audio, or Transcripts, other than (i) transient caching necessary for real-time display and the operation of your application, (ii) retaining a response you received, unaltered, so that your users can view that prior response…

(h) Reuse stored Podcast Data to answer a later or different query. Each query is served by a new request through the Services.

(i) Where an Automated Agent handles Podcast Data, allow it to retain that data beyond the session context or short-term memory reasonably necessary to process the immediate request. No persistent storage by agents.

(a) Redistribute, resell, sublicense, or make Podcast Data available to anyone other than your own authorized users…

Go back through the plan with those in hand.

The SQLite table is out. Saving transcripts is (g). Saving episode rows and comparing them next week is (h), almost word for word: stored data answering a later query.

The agent workflow is out. Every post in this series was built with a coding agent driving the CLI. Under (i), the agent can hold the data for the request and that’s it.

This blog post is out, mostly. You are not one of my authorized users. A table of which shows mentioned which company, in a public post, is (a).

There is also (j), no training models on it, and section B.7, which says Particle “may not hold licenses from the podcast creators and publishers” for the content. Particle’s position is fair use, and the schedule says that the podcast owners “may not share that assessment.”

I Read the Terms for Every Paid Provider

If the podcast provider looks like that, what about the rest? Alexandria has a terms catalog, and terms show works on any provider ID, so I walked the categories and asked about every provider I could find.

Most of them have no terms in the catalog at all. Government data, public registries, app stores, package registries, Yahoo Finance. terms show returns “Provider not found in the accessible terms catalog.” That doesn’t mean no rules. It means Firecrawl has no extra contract for that source, and the source’s own terms of service apply the way they would to any scrape.

The commercial partners each carry a lettered schedule. Seven of them:

Provider	Data	Store it?	Publish it?	Accept first?
Similarweb (G)	Website traffic	No. Not even cached. Each data point answers only the query that fetched it, in that session	To your users, one query at a time, with “Insights by Similarweb”, their logo and a link	No
Particle (B)	Podcasts	No , beyond an unaltered past response	No	Yes
Apollo (F)	Contacts, companies	Yes, for your own sales, marketing and recruiting	No , not even through an agent’s output	Yes
FullEnrich (A)	Contacts	For your own sales, marketing and recruiting	No	No
Data Legion (H)	People, companies	Only in your CRM and internal systems	No (your own analyses are yours)	No
Benzinga (C)	Market news	As needed to serve your own product	Inside your product, to your users	No
Fiscal.ai (D)	Financials, filings	Yes, within your apps	Inside your apps, to your users	No

“Accept first?” is the required flag. When it says No, the call goes through without an acceptance step. The schedules still describe how you are allowed to use the data, and nothing in them suggests skipping the acceptance step lets you skip the rules.

So it sorts into three piles. The contact providers are built for your sales team and forbid publishing. Similarweb and Particle forbid keeping anything. The financial providers leave you room to build a product.

None of this is Firecrawl being sneaky. Licensed data has always come with rules like these. What changed is that it now sits behind the same firecrawl scrape you use on a public web page, where the only rules are robots.txt and the site’s terms. One command, two completely different legal situations, and nothing in the syntax tells you which one you are in.

The Tracker That Survives the Terms

The thing (h) forbids, keeping your own history to answer later questions, is a thing Particle sells you instead. podcasts/mentions/timeseries costs 15 credits and returns the whole history, bucketed by day, week or month, in one call. You don’t need a database if every run asks for everything.

I pointed it at Anthropic, monthly, since January. The episode counts climbed steeply all year. It looked like a story. Then I ran the same call on Coca-Cola as a control (a company that didn’t get more famous in 2026), and it climbed steeply too.

That is not Anthropic or Coca-Cola. That is Particle’s index growing. More shows transcribed, more episodes covered, more mentions of everybody. A raw mention count from a source whose coverage changes every month will tell you your brand is exploding when the source is the thing exploding.

The fix is old: divide by a control. Same source, same window, a company whose real share of the conversation should hold steady. Whatever index growth inflates in one, it inflates in the other.

The second gotcha: don’t pass a domain

company_id says it takes a slug, a domain or an ID. anthropic.com worked. coca-cola.com came back as Simply Orange , one juice brand out of Coca-Cola’s two hundred, with a handful of mentions. No error, no warning. If I hadn’t looked at the company block in the response, I’d have used a juice brand as my control and believed the result.

So the script resolves names first with entities/search (5 credits) and takes the exact match.

The script

No SQLite, no saved files. It prints to the terminal and forgets:

#!/usr/bin/env python3
import json, subprocess, sys

def call(tool, options):
    out = subprocess.run(
        ["firecrawl", "scrape", f"particle/{tool}",
         "--options", json.dumps(options), "--json"],
        capture_output=True, text=True, check=True).stdout
    return json.loads(out[out.index("{"):])["data"]["alexandria"][0]["data"]

def company_slug(name):
    # Never pass a bare domain: coca-cola.com resolved to Simply Orange.
    hits = call("entities/search", {"q": name, "type": "company", "limit": 5})["data"]
    exact = [h for h in hits if h.get("match_quality") == "exact"] or hits
    if not exact:
        sys.exit(f"no company found for {name!r}")
    company = exact[0]["company"]
    print(f"{name!r} -> {company['name']} ({company['slug']})")
    return company["slug"]

def monthly(slug, since):
    data = call("podcasts/mentions/timeseries",
                {"company_id": slug, "interval": "month", "published_after": since})
    return {b["start"][:7]: b["episode_count"] for b in data["buckets"]}

def main():
    target, control = sys.argv[1], sys.argv[2]
    since = sys.argv[3] if len(sys.argv) > 3 else "2026-01-01"
    t = monthly(company_slug(target), since)
    c = monthly(company_slug(control), since)
    print(f"\nmonth {target} episodes per {control} episode")
    for month in sorted(t):
        ratio = t[month] / c[month] if c.get(month) else float("nan")
        print(f"{month} {ratio:6.2f}")

if __name__ == " __main__":
    main()


python3 podcast_share.py "Anthropic" "The Coca-Cola Company"

Each run is two searches and two timeseries calls: 40 credits. Monthly, that’s 4% of the free 1,000. Weekly, about 17%. Daily is 1,200, which is more than the free plan.

What I can show you

And (a) bites the post itself: Particle’s counts are Podcast Data, so I’m not printing them. The ratio is a number I computed from them, which I think puts it on my side of the line. I am not a lawyer, and you should make that call for your own use, not borrow mine.

month Anthropic episodes per The Coca-Cola Company episode
2026-01 1.57
2026-02 2.94
2026-03 3.87
2026-04 3.27
2026-05 3.15
2026-06 3.57
2026-07 2.78
2026-08 3.14
2026-09 4.89
2026-10 3.89

Read that against the raw curve, which climbed all year. Corrected for index growth, Anthropic’s share roughly doubled from January to February and then sat around three for six months. September is the only real spike. October is a partial month as I write this. The steep line was mostly Particle getting bigger.

Read the Schedule Before You Write the Recipe

The rule from the Alexandria post was check the catalog before you write a scraper. Universal Scrape adds a step after that: check the terms before you write anything that stores, compares or publishes.

The check is one free command per provider:

firecrawl alexandria terms show <provider> --json

If it says the provider is not in the terms catalog, you’re in normal scraping territory, with the site’s own terms and everything the series has said so far. If it returns a schedule, search it for “store”, “retain”, “available to” and “agent” before you design anything. Those four words decide whether your idea is a cron job, a dashboard for your own team, or nothing.

My podcast tracker ended up as a 40-line script that forgets everything it learns the moment it prints. That’s less than I wanted. It’s also the only version I can actually run, and it gets right what the version I planned would have gotten wrong: the growth curve was mostly the index.

The financial providers are the ones whose terms let you build something real, which makes Fiscal.ai the next thing I want to look at.

Eleven Deep Research Tools That Live Inside Your Coding Agent, and the One I Forked

Stephan Miller — Wed, 14 Oct 2026 12:00:00 +0000

The first time I ran my research agent on a real topic, it ran for over an hour and I had no idea whether it was working or hung. Nothing between “started” and whatever it was eventually going to print. So I sat there watching a spinner.

It was not almost done, because it had no definition of done.

That agent was a fork of somebody else’s project, and before I spent another weekend modifying it to my needs I wanted to know whether I should just use one of the tools that already existed. So I took a deep look at the other tools. Eleven tools that run inside a coding agent, or that an agent can drive.

I put the breakdown in my own repo as ALTERNATIVES.md.

This post is that page with the story attached.

One Way to Rank These
The Eleven
- The Report Writers
- The Ones That Return a Table Instead of Prose
- The Ones That Hand You Sources, Not Conclusions
- The Audit Layer
- And STORM, Which Is a Reference Design Now
The One I Forked
- It Would Not Install the Way I Wanted
- The Agent With No Tool Allowlist
- No Budget, No Stop Condition, No Way to Tell Working From Stuck
- The Part I Really Liked: Search Modules
- Building a Module by Discovering Its Sources Instead of Guessing Them
- Installing It Does Not Give You a Tool, It Starts a Research Project
- Everything Else That Got Bolted On
What It Is Bad At
Using It
What I Actually Took Away From This

One Way to Rank These

One instinct with research tools is to rank them by depth. Which one goes deepest. Which one reads the most sources. Which one runs the longest.

Depth is a dial that almost all of them have, so depth is not something to compare here.

The two properties that actually decide which tool you want are shape and persistence.

Answer shape. A run produces one of four things. A narrative document, which is chapters and prose and citations. A matrix, which is N subjects by M attributes, also known as a comparison table. An annotated source list, which is findings with provenance where you do the synthesis. Or a knowledge base, which is a corpus that grows with new research instead of a single deliverable that ships.

Persistence. A run is either ephemeral, where the findings exist once and the next run starts cold, or accumulating, where there is a store that gets consulted before anything gets fetched. An accumulating tool used twenty minutes a week beats an ephemeral one used eight hours once, if the topic is something you keep coming back to. Which, if you’re researching your own industry, it is.

Two smaller axes worth checking before you commit:

Source routing. Does it search the open web generically, or does it know where the answers for a given kind of question actually live? Generic width does badly on any topic where vendor marketing and programmatic SEO own the first hundred results. Try researching LLM pricing with a generic web sweep sometime and count how many of the top results are listicles written by other AI.

Provider lock. Is it welded to one model vendor, or can it run on whatever you have, up to and including a local model?

The Eleven

Everything below runs inside a coding agent or is something an agent can drive. Skills, MCP servers, CLIs, libraries. I deliberately left out the hosted deep research products, the ChatGPT and Gemini and Perplexity tier, because their output lives in somebody else’s UI. The entire reason to run research from an agent is that the findings land in your own files.

Stars and last-push months here were verified against the GitHub API on 2026-10-10. Everything else in this section is taken from each project’s own documentation. I haven’t independently reproduced anybody’s benchmark placement or run time.

Tool	Form	Answer shape	Persistence	Stars	Last push
local-deep-research	MCP + lib + CLI	Narrative report	Encrypted SQLite history + your own docs	9.2k	2026-10
hyperresearch	Claude Code skill + MCP + CLI	Narrative, chaptered	Markdown vault + SQLite index	3.8k	2026-10
gpt-researcher	Python lib (+ MCP)	Narrative report	None	30k	2026-10
last30days-skill	Claude Code skill	Ranked brief	SQLite archive of past briefs	63.9k	2026-10
paper-qa	Python lib + CLI	Cited answer	Reusable index under `~/.pqa/`	9.3k	2026-09
local-deep-researcher	LangGraph app	Summary + sources	None	9.4k	2026-09
deep-searcher	Python lib + CLI	Narrative report	Milvus vector DB	8.3k	2026-09
DeepGit	MCP + CLI + lib	Ranked matrix	LanceDB vector index	919	2026-08
hoolulu/deep-research	Cross-platform skill	Narrative, chaptered	Folder of dated reports	598	2026-08
reddit-research-mcp	MCP server	Cited findings	Saved feeds	254	2026-09
RivalSearchMCP	MCP server	Structured JSON	None	132	2026-10

The Report Writers

local-deep-research is the most complete package in the table. Per-domain engines for arXiv, PubMed, Semantic Scholar, NASA ADS, GitHub, SearXNG and the Wayback Machine. An encrypted SQLCipher history of every run you have ever done. A personal document library that gets indexed and searched right alongside the live results. It runs fully offline on Ollama or llama.cpp if you want it to. Use this one when the corpus is what you want and you’ll be coming back to the same subject for months.

hyperresearch is a sixteen-step pipeline with three tiers, and the top tier, dissertation, is opt-in only. It partitions your topic into four to ten chapters and then loops the research steps once per chapter. That’s how you get 25,000 to 80,000 words out of it. Markdown is the source of truth and SQLite is the cache, so the vault stays portable and greppable. It’s Claude-only and it puts Opus on the synthesis and critique roles, so a long run isn’t cheap. Reach for it when you want one large chaptered document at the end.

gpt-researcher is the mature baseline. Thirty thousand stars and the one most likely to already be supported by whatever you’re building on top of. Its MCP surface, specifically get_research_context and get_research_sources, hand back the raw material separately from the written report. That means your agent can do the writing instead of getting handed prose. One caveat: you need a paid search API to get good results out of it.

hoolulu/deep-research produces the same chaptered shape as hyperresearch but it’s portable across Claude Code, Codex, Cursor, OpenCode, Windsurf and Cline, and it is multilingual.

local-deep-researcher is the smallest readable implementation of the reflect-then-fill-the-gap loop that all of these are variations on.

deep-searcher inverts the whole premise: research over your private corpus, with the web as an optional supplement. That’s the opposite of everything else on this list.

The Ones That Return a Table Instead of Prose

DeepGit does GitHub-only tool discovery and returns a scored comparison table. It preserves your actual query intent, meaning must-haves, nice-to-haves and anti-patterns, instead of flattening everything into keywords. This is ethe right tool for “find me the N projects that do X,” which is the question this entire post is an answer to.

last30days-skill is recency-windowed research across social and forum sources. Reddit, X, Hacker News, YouTube transcripts, Bluesky, Polymarket odds. It ranks by actual engagement, not search position, so it answers “what are people saying about this right now.” Add --store and the briefs accumulate into a local SQLite database you can run scheduled watchlists and digests against.

The Ones That Hand You Sources, Not Conclusions

paper-qa does academic literature with citations traceable down to page ranges, across your own PDFs plus Semantic Scholar, Crossref and Unpaywall. Use it when a claim has to be defensible, not just plausible.

reddit-research-mcp is semantic search across Reddit where every finding cites a real post or comment, upvote count included. This is where you get demand signals and pain points in people’s own words. Generic web search flattens all of that into blog posts about the topic, written by people who also only read blog posts about the topic.

The Audit Layer

RivalSearchMCP has no LLM inside the server at all. Every result carries a tier, freshness, corroboration and citation score out of 100, every multi-result response carries an aggregate confidence signal, and find_conflicts surfaces numeric, date and polarity disagreements between sources instead of averaging them. It is an auditing layer instead of a research tool, and it composes with any of the others.

And STORM, Which Is a Reference Design Now

STORM out of Stanford did not make the table, but it deserves the mention because it’s the only project in this space whose output shape is a wiki article. It gets there by discovering perspectives from existing articles on similar topics, then simulating a conversation between a Wikipedia writer and a topic expert grounded in real sources. Which is a clever way to get an outline that is not just the first five headings an LLM associates with your topic. Last pushed 2025-09. Treat it as a reference design to steal from, not a dependency to take on.

The One I Forked

The pipeline in deep-research-agent is not my design. The skills, the agent and the original strategy modules come from Weizhena/Deep-Research-skills by Lan Zheng, MIT licensed, and all the credit for the structure goes there. Outline, then fields, then one agent per item, then roll the JSON up into a report. I kept that part verbatim.

It Would Not Install the Way I Wanted

APM only accepts a source whose root is a package, meaning a SKILL.md, or an apm.yml plus .apm/, or a plugin.json. Upstream ships its agent as a bare agents/ directory with none of those, so apm install Weizhena/Deep-Research-skills/agents fails validation. Which is a problem, because the skills don’t work without the agent.

So the first thing that happened was repackaging the whole thing into one APM installable bundle:

apm install eristoddle/deep-research-agent --target claude
# or
apm install eristoddle/deep-research-agent --target copilot

The Agent With No Tool Allowlist

Upstream’s agent has no tools: frontmatter at all, which means it inherits every tool the host offers. Under Copilot in VS Code, that includes the embedded browser and the download tooling.

An agent whose instructions say “systematically explore” roughly twenty five source families, handed a browser, opened about a hundred editor tabs in a single run. In the same run it decided to write its own report-generator script rather than use /research-report.

It’s now restricted to WebSearch, WebFetch, Read, Write, Bash with an explicit prohibition on browser automation, downloads, and self-authored research scripts.

No Budget, No Stop Condition, No Way to Tell Working From Stuck

Back to my spinner. Upstream tells the agent to generate “5-10 query variations,” to “read beyond the first few results,” and to cover that merged list of about twenty five source families. There is no cap on searches, no cap on fetches, no cap on link depth, and no definition of done.

There is now a three-level depth setting, defaulting to standard:

Level	Searches	Fetches	Link depth	Modules
`quick`	3	4	1	1
`standard` (default)	8	12	1	2
`deep`	20	30	2	3

Set it per run with /research-deep deep, per project with execution.depth in outline.yaml, or ad hoc by naming explicit numbers in the task prompt. Explicit numbers win, then the named level, then the outline, then the default.

The agent now reports budget state as it goes, like [deep: 3/20 searches, 5/30 fetches], and it stops at the ceiling. The budget also propagates down to every per-item agent, along with the tool prohibitions, so the constraints survive the handoff. And the fetch budget counts every retrieval attempt.

Instructions in the prompt that contradicted the budget got rewritten, not left in there to fight it. That sounds obvious. It’s also the step everyone skips when they bolt a limit onto an existing prompt, and then they wonder why the model ignores the limit.

The Part I Really Liked: Search Modules

Search modules are what make this repo worth having instead of just a patched fork.

Most research tools search the open web generically and then scale by adding more sources. This one tries to search fewer, better places, and it decides where to look before it searches anything. That’s what a search module is: a per-domain routing file with a source list, an access method per source, an ordering rule for which sources outrank which, and documented workarounds for the ones that block agent fetching.

There are twelve of them now. Five are upstream’s, and those five assume you are debugging code or reading papers: general-web, github-debug, stackoverflow, academic-papers, chinese-tech.

Now there is also an AI ecosystem and market family:

benchmarks puts independent evaluators first, makes the configuration and observation date mandatory, and labels self-reported scores as self-reported.
model-releases ranks docs and deprecation pages above announcement posts, and wants model IDs, not marketing names.
pricing will not quote a price from anything but the provider’s own page, and records unit, host and date with it.
vendor-landscape builds the competitive set from vendors’ own comparison pages, and keeps claim separate from evidence.

Plus agent-tooling, which is its own family and covers the things you install and run, not call over an API. Agent CLIs and IDEs, MCP servers, skills, plugins. The questions you actually have about those are the config format, the permission model and the version history.

Then two more that are not about technology at all. competitor-content samples what has already been published on a topic, reading three to five top pages for outline, not prose, plus People Also Ask and forum questions, and reports the union of subtopics and the gap. It’s also explicit about its limits: WebSearch returns a result list, not a SERP, so ranking position, search volume and keyword difficulty are unavailable. I would rather a module tell the agent “you can’t know this” than let it produce a confident number out of nothing.

demand-signals answers “what do people keep complaining about, in their own words, across places that do not talk to each other.” That needs raw user language, not a ranked page or a spec sheet.

Building a Module by Discovering Its Sources Instead of Guessing Them

/research-add-module makes you do the source discovery for real. Draft the questions the module has to answer. Run them as plain general-web queries. Tally which domains actually held the answers. Then work out and test how to query each one, whether that’s a site: query, the site’s own search URL, a stable index page you can fetch directly, or a note recording that it blocks fetching entirely. It registers the result, then checks whether the new module actually beats general-web on the original questions, and offers to throw it away if it doesn’t.

demand-signals got built that way, by running the discovery pass across seven unrelated probe topics instead of writing a source list that looked right. What came out of it was more specific than anything I’d have guessed. Discourse’s search.json works and I confirmed it on two unrelated instances. Other forum software falls back to a site: query, because XenForo’s own community confirms its search is POST-only with no fetchable route. People Also Ask earns its own dedicated query pass. Amazon product Q&A went in as unreachable, 503 on every product I tested, recorded rather than silently dropped.

Installing It Does Not Give You a Tool, It Starts a Research Project

You do not install this and end up with a research agent. You install it into a project, and from that point the copy inside that project starts drifting toward the kind of research that project does. Three mechanisms do it.

Runs accumulate into a tree, not a pile. Every run lives under one root, research/ by default, and research/INDEX.md records a tree, not a list. Research does not happen once. One run’s findings point at the next one, so the index keeps a map of which run spawned which, plus a **Leads** checklist per run of the directions that run opened and I haven’t chased yet.

Sources that keep earning their place get promoted. /research-harvest scans every run under the root and tallies which source URLs keep showing up. A URL becomes a promotion candidate once it has appeared in at least three runs and actually supported a field in at least two of them. It never writes into a module itself. It writes a candidate list for a human to choose from.

The modules it feeds live in the project and survive reinstalls. Local modules go in .agents/web-search-modules-local/, which APM doesn’t own. The router checks that directory before its own table and the local module wins a name conflict.

Put those three together and you have a loop. Run research, find out which sources actually answered something, promote the repeat domains into a module that lives in this project, and the next run here routes better than the last one did. A client repo ends up knowing that client’s competitive set. The repo where I research AI tooling ends up knowing where the real leaderboards and deprecation pages are and ignoring everything else.

Everything Else That Got Bolted On

Routing lives in a file, not in the prompt. Upstream hardcodes five trigger-to-file lines into the agent’s system prompt. That works at five modules and falls over as the library grows, because overlapping triggers get resolved by dropping one, and every module costs system-prompt context on every task whether or not it’s relevant. Routing now lives in skills/web-search-modules/ROUTING.md, which the agent reads before any search. Adding a module is one file plus one row and the prompt never changes.
A bounded fetch escalation ladder. WebFetch returns 403s and bot challenges and JS-only shells often enough to lose real sources. The agent can escalate one URL through crwl (crawl4ai) if it is installed, then Firecrawl if both the CLI is on PATH and FIRECRAWL_API_KEY is set. Every rung is stdout-only and bounded, no install attempts, and the whole ladder doesn’t buy an extra fetch slot because it’s one logical fetch of the same page. I wrote about that whole mess separately in What to Do When Your AI Coding Agent Can’t Read a Web Page.
unreachable[] is its own output channel. A page nothing could fetch is provenance about a source. It’s not a reason to mark a field uncertain when that field got answered through a documented substitute. The report renders an ## Unreachable sources section by default.
/research-enumerate is for when the hard part is finding the items at all. The pipeline only had one shape of work, which was “here are the items, go deep on each.” That doesn’t answer “what exists in this space” when each item needs three shallow facts and the difficulty is enumeration. It sweeps aggregators first, the directories and registries and awesome-lists that list the leaves, before going after the leaves themselves. Its real discipline is the coverage declaration: a category the sweep never reached gets its own subsection saying so and naming the exhausted budget as the reason, never silently omitted, because an unreached category that goes unmentioned reads as a negative result to every later reader.

What It Is Bad At

This repo produces a matrix. You define items and fields, it researches every field for every item with one agent per item, and the report rolls the values up. It is a good fit for “these twelve models across these fifteen attributes.” It is a bad fit for “write me a long document about this topic,” and there’s no reason to force it into that job when six tools listed in this post do it better.

It’s also human-in-the-loop on purpose. The outline gets reviewed and amended before any research runs. That costs you a step. What it buys is the ability to fix a bad research plan before you spend an hour executing it.

And field counts matter a lot. A 42-item outline with 96 fields is roughly four thousand researched values and it will not finish. Keep field lists to fifteen or twenty and prefer items_per_agent: 1, so each finished agent yields a complete file instead of an all-or-nothing batch. Ask me how I learned that one.

Using It

/research <topic> # interactive: builds outline.yaml + fields.yaml
/research-enumerate # breadth-first: catalog what exists into catalog.md + outline.yaml
/research-add-items # amend the item list
/research-add-fields # amend the field list
/research-add-module # build a search module for a domain the existing ones miss
/research-deep # one agent per item -> one JSON per item in results/
/research-report # roll the JSON files up into report.md
/research-harvest # hand-invoked: tally recurring sources across runs

/research and /research-deep pick the host-native entry point. Claude launches web-search-agent, Copilot launches Web Research Writer. The skills do not work without the packaged agents, which is the entire reason the bundle exists.

What I Actually Took Away From This

Most of the work in that fork was not features. Count them up and the majority of the commits are guardrails: a tool allowlist, a search budget, a stop condition, a rule about what counts as a fetch, a prohibition on writing your own scripts, and a routing file so the prompt stops growing. The pipeline design was already good. It’s the reason I forked it.

I will do the ALTERNATIVES page again. Writing “here are eleven tools, nine of them beat mine at something, here is the one job mine is actually for” clarified the repo more than any feature I added.

Also, the part I worked on the most is a part I never planned. I built a matrix generator. Then an index, because run folders were cluttering up repos where I was also writing code. Then a harvest command, because I kept hand-copying the same three URLs into modules. The loop those three turned out to make showed up on its own and became the reason to keep using the thing.

Claude Haiku 5.5 Is 90% Cheaper Until Your Prompt Hits 100K

Stephan Miller — Tue, 13 Oct 2026 13:00:00 +0000

Anthropic now sells a model for the same 50 cents per million output tokens as GLM-5.3-Flash.

That’s the company whose Fable 5.1 costs $50 out. GLM-5.3-Flash is the Chinese open-weights model that’s been anchoring my cheapskate table for six weeks while everybody with a bigger brand looked down at it. On October 7, Claude Haiku 5.5 moved into the same price bracket and scored a point higher on Artificial Analysis. I was ready to write the “Anthropic finally gets it” post. Then I read the pricing page all the way to the bottom. You know how that goes.

Same week, Meta went the other way. Its first real frontier model, Muse Wonder, launched October 12 at exactly what Claude Opus 5.5 costs, and it scores lower than Opus. And the free stealth model that was number one on OpenRouter last week disappeared on October 5. Its traffic didn’t go to a better model. It went to the next free one. So this is a read-the-footnotes week. The sticker price on almost every model that launched lies to you somewhere.

Claude Haiku 5.5 moved into GLM’s neighborhood
- The 100K trapdoor
- It’s chatty
- Arena hasn’t rated it yet
Meta built a frontier model and priced it like Opus
Space Bunny left, and the free traffic found a new home
- DeepSeek V4.1 Flash got the paying customers
- Step 5 Preview got the freeloaders
- One more launch: Mistral Large 4
The cheapskate table: Argon still owns the ceiling, and the edges got sharp
Horror stories
- The 100K trapdoor
- GitSpawn: your repo’s git config can run code
- PixelLeak: the agent proved its work in public
- The free week ends Friday
What’s coming
Finally

Claude Haiku 5.5 moved into GLM’s neighborhood

Here’s the pitch, and most of it holds up. Claude Haiku 5.5 costs $0.10 in and $0.50 out per million tokens. Haiku 4.5 was $1 and $5, so per token that’s a 90 percent cut. It gets a 1M context window (Haiku 4.5 had 200K), 128K output tokens, and the effort parameter, which no Haiku has had before. Adaptive thinking is on by default, so it decides for itself when to think.

On Artificial Analysis, Haiku 5.5 at max effort scores 43. GLM-5.3-Flash scores 42, Gemini 3.8 Flash 41, and GPT-6 Luna 38. That makes it the smartest model at the 50-cent floor. It’s also fast. AA clocks it at about 237 tokens a second, almost four times GLM-5.3-Flash’s 62. And AA puts it in the good corner of its intelligence-versus-cost chart.

On OpenRouter it did 1.35 trillion tokens in its first week, second on the New & Trending list. Anthropic’s share of OpenRouter requests went up 59 percent. That’s still only 3.9 percent of requests, but it’s moving.

So why am I not putting it in the cheapskate table? Three reasons, and the third one is my own rule.

The 100K trapdoor

Haiku 5.5’s price has a cliff at 100,000 prompt tokens. Under the line you pay $0.10/$0.50. Over it you pay $0.50/$2.50, and the higher rate applies to the whole request, not just the tokens past the line. eesel did the math: a 90K prompt with 1K of output costs $0.0095. Make it 110K and it costs $0.0575. That’s about six times the price for 22 percent more tokens. Who thought this up? Cached tokens count toward the line too, so caching doesn’t save you from crossing it.

And you’ll hit the line sooner than you think. Haiku 5.5 uses the newer Claude tokenizer, so the same text comes out about 30 percent more tokens than on Haiku 4.5. Anthropic’s own migration notes say that, and they also say Haiku 5.5 keeps thinking blocks from every earlier turn in context, billed as input. Haiku 4.5 only kept the latest turn’s. So a long agent conversation grows faster on 5.5, and agent conversations are exactly where you hit 100K.

It’s chatty

Thinking tokens are output tokens, and Haiku 5.5 thinks a lot. AA’s page shows it used 440 million output tokens to get through the Intelligence Index. Two issues ago I wrote about Sonnet 5.5 at max effort costing more per task than Opus because it burned 410 million. The small Claude now out-talks the medium one. At 50 cents out it’s still cheap in absolute terms. But “cheap per token” and “cheap per job” are different numbers, and with Claude you have to work out the second one yourself. There’s one more gotcha in the docs: thinking tokens count against max_tokens. If you’re migrating code with a small limit, you can get a response that’s all thinking block and no answer.

Arena hasn’t rated it yet

My cheapskate method only picks models with an Arena rating, and Haiku 5.5 doesn’t have one yet. Next week it probably will. If it lands within 50 points of the leader in any category, it ties GLM-5.3-Flash on price, and then it gets interesting.

If your prompts are short (classification, routing, extraction, subagent grunt work), Haiku 5.5 might be the best 50-cent model you can buy, and it’s fast. If your prompts are long agent histories, check your token counts on the new tokenizer before you switch anything.

Meta built a frontier model and priced it like Opus

Meta’s Muse Wonder launched Monday, along with a bigger sibling called Muse Wonder Full. It’s the first thing Meta has shipped since rebuilding its AI group that it’s willing to line up against Fable 5.1 and GPT-6 Astra. They’re agent models: they plan, hand work off to sub-agents, call tools in parallel, and take text, images, video, audio, and PDFs with a 1M context window.

The price is $4 in, $0.20 cached, and $20 out. Claude Opus 5.5 costs the same, to the penny. Meta’s Alexandr Wang had said API pricing would be “meaningfully below” the rivals (Implicator has the quote). It isn’t.

On Artificial Analysis, the Muse Wonder you can actually call through the API scores 52. Opus 5.5 scores 58. Muse Wonder Full scores 56, but Full isn’t on the API. You get it through a $50 Muse Code plan. So the model with the Opus-level score is behind a subscription, and the one with the Opus-level price scores six points lower.

Meta’s own chart has Wonder Full winning 7 of 14 rows, including 57.5 on something called SWE-Marathon against Opus 5.5’s 46.3. Read the footnotes, though. Wonder Full ran that one at xhigh effort while the other models ran at max, and the model card itself says the row “is not a controlled common-harness comparison.” Meta also calls its competitor runs “best-effort” and admits the prompts “may not be optimally tuned for proprietary models.” I’ve seen that move before. Meta graded the other team’s homework, and you should read the chart that way until somebody independent runs the numbers.

The weights are closed, so this isn’t the Llama-era Meta. And Meta already showed us its pricing style in August with the Muse Code “contributor” tier, where you got a discount in exchange for letting Meta train on your prompts. This time there’s no discount. It’s not on Arena yet either, so no cheapskate math. My read: if you already pay Opus prices, there’s no reason to switch to the model that scores lower for the same money. If Wonder Full ever gets an API at $20 out, that’s a different conversation.

Space Bunny left, and the free traffic found a new home

Last week I said that when Space Bunny Alpha’s free ride ended, we’d find out how much of its 38.5 trillion tokens came from people who actually wanted the model. The free ride ended October 5. OpenRouter posted that the stealth period was over and the model “will be revealed soon.” It still hasn’t been. The best guess is still MiniMax M3.1-Flash-Preview, and that’s still unconfirmed. Space Bunny is number two for the trailing month at 56.7 trillion tokens. It doesn’t show up in this week’s top 10 at all.

So where did the traffic go? Two places.

DeepSeek V4.1 Flash got the paying customers

OpenRouter’s weekly board through October 12 has DeepSeek V4.1 Flash at number one with 40.5 trillion tokens, up 43 percent from 28.4 trillion last week. It’s also number one for the day and the month (109 trillion). DeepSeek is now 24.5 percent of all OpenRouter requests. Z.ai, the company behind GLM, more than doubled its requests (up 107 percent, to 13.4 percent).

If you have to start paying, that’s not a bad place to land. DeepSeek charges $0.15/$0.60 direct, AA gives it 39, and it pushes about 199 tokens a second, which is more than three times GLM-5.3-Flash’s speed. A lot of Space Bunny’s users apparently did the math and decided 60 cents was close enough to free.

Step 5 Preview got the freeloaders

Number three this week is a model most people hadn’t heard of: StepFun’s Step 5 Preview, at 12.1 trillion tokens in its first week on OpenRouter. It’s a 600B-parameter mixture-of-experts with 27B active, and it lists at $1 in and $2.70 out.

Nobody pays $2.70 out for 12 trillion tokens in a week for a model they’d never heard of. So I looked. According to OrcaRouter’s tracker, Step 5 Preview went free in three agent tools within five days: Cline on October 8 for a week, Nous Portal from October 8 through 15, and Kilo Code from October 9 for a week. Nous Portal is the API from Nous Research, the same people behind Hermes Agent, which is OpenRouter’s busiest app at 1.99 trillion tokens a day. Kilo Code is the third busiest. The free price goes back to $1/$2.70 on October 16.

Arena has it at 83rd Overall and 71st in Coding. AA is kinder and gives it 44, about the same as Kimi K3, but notes it’s verbose: 160 million output tokens on the index against a median around 82 million. StepFun says open weights are coming October 15. As of this morning, the newest thing on StepFun’s Hugging Face page is Step-3.7-Flash from June.

This is the free-promo spike I keep warning about. A model in the top three by tokens and 83rd by votes is a model people are using because it costs nothing this week. Use it for free while you can, sure. Just don’t read 12 trillion tokens as a recommendation. Check back on the 17th.

One more launch: Mistral Large 4

Mistral put Large 4 into public preview on October 6. It’s 1.05 trillion parameters, 49B active, with open weights scheduled for October 31. OpenRouter lists it at $0.68/$2.09, AA gives it 38, and Arena has it 115th Overall but 30th in Math on 220 votes. It’s not a cheapskate pick yet. It’s here because a European lab putting out trillion-parameter open weights would matter, and I want to see if they land on time.

The cheapskate table: Argon still owns the ceiling, and the edges got sharp

If you’re new here: for each Arena category I take the leader’s rating, subtract 50, and look for the cheapest model still above that line. The band gets computed in code from every row on the board, and I write down the first model that falls below the line so I know nothing got cut off. (I learned that last part the hard way in July.)

This week arena.ai decided I was a bot. Cloudflare served me a “Sorry, you have been blocked” page from the browser, from the fetch tool, and from plain curl, which is fair, because I kind of am one. So I pulled the board from LMArena’s own leaderboard dataset on Hugging Face instead. It’s the same style-controlled board the website shows (Argon at 1525 Overall, same as last week), published October 8. Every row is in there, which is what the method needs. There’s no price column, so I priced each model off its own maker’s endpoint on OpenRouter. That’s an improvement, actually. For weeks Arena’s price column kept showing whichever third-party host was cheapest that day, and I kept having to correct it.

Gemini 4 Argon is still number one in all six categories, still on preliminary votes (271 in Math), and still not for sale outside Google’s Fairwind program. Its $10 out is an intro price that goes to $20 with no end date announced.

Category	Leader	$ out	Cheapskate pick	$ out	Δ rating	Cheaper by
Overall	gemini-4-argon-high (1525)	$10	MiMo-V2.6-Pro (1480, #26)	$0.87	−45	11.5×
Coding	gemini-4-argon-high (1562)	$10	MiMo-V2.6-Flash* (1514, #45)	$0.28	−48	36×
Creative Writing	gemini-4-argon-high (1519)	$10	gemini-3.7-flash-high (1492, #5)	$3.75	−27	2.7×
Instruction Following	gemini-4-argon-high (1529)	$10	gemini-3.8-flash-high (1489, #14)	$3.75	−40	2.7×
Hard Prompts	gemini-4-argon-high (1551)	$10	MiMo-V2.6-Pro (1506, #23)	$0.87	−45	11.5×
Math	gemini-4-argon-high (1528)	$10	GLM-5.3-Flash (1502, #13)	$0.50	−27	20×

*1,989 votes, 2.4 points above the cutoff. GLM-5.3-Flash sits at 1525 with 7,267 votes for 50 cents, and it’s the safer bet.

Band depths: Overall 32, Coding 51, Creative 18, Instruction Following 20, Hard Prompts 29, Math 40.

And the version you can actually buy from, with Argon removed:

Category	Purchasable leader	$ out	Cheapskate pick	$ out	Cheaper by
Overall	claude-opus-5.5-high (1507)	$20	GLM-5.3-Flash (1475)	$0.50	40×
Coding	claude-fable-5-high (1551)	$50	MiMo-V2.6-Flash (1514)	$0.28	179×
Creative Writing	claude-opus-5.5-high (1517)	$20	gemini-3.7-flash-high (1492)	$3.75	5.3×
Instruction Following	claude-opus-5.5-high (1519)	$20	GLM-5.3-Flash (1472)	$0.50	40×
Hard Prompts	claude-opus-5.5-high (1533)	$20	GLM-5.3-Flash (1501)	$0.50	40×
Math	claude-fable-5-high (1524)	$50	GLM-5.3-Flash (1502)	$0.50	100×

(In Hard Prompts, MiMo-V2.6-Flash sits right at the edge of the band for 28 cents. I’ll take the GLM row with 17,734 votes over a model sitting 0.3 points above the line.)

Opus 5.5 finally took the purchasable Overall spot from Opus 4.6, a model from February. The picks themselves barely moved. What changed is how close to the line everything is sitting.

GLM-5.3-Flash missed the Hard Prompts band by 0.3 points. The cutoff is 1501.0 and GLM is at 1500.7. That’s the whole reason MiMo-V2.6-Pro is the Hard Prompts pick at 87 cents instead of GLM at 50. If Argon’s rating slips by a third of a point, or GLM gains that much, it flips.

Coding’s band edge is basically a tie. The first model below the Coding cutoff is GLM-5.1, short by two hundredths of a point. And MiMo-V2.6-Flash, the cheapest model in the band for the third week running, dropped from 1521 to 1514, with only 1,989 votes. My rule for it was that if it held the spot on a few thousand votes, I’d call it the Coding pick for real. It’s going the other way. GLM-5.3-Flash is still the safer bet.

Last week’s Instruction Following pick, DeepSeek V4.1 Flash, missed the band by 2.2 points this week. With it gone, the cheapest thing left in the band is Gemini 3.8 Flash at $3.75. So Instruction Following joins Creative Writing as a pay-for-quality category, though $3.75 is still 2.7 times under Argon’s intro price.

And a free model snuck into the ex-Argon Coding band. Google’s Gemma 4 31B is at 1502.7 against a 1501.4 cutoff, and OpenRouter has a free variant (34 cents out paid). It’s barely in, and a free endpoint comes with free-endpoint terms (read what they do with your prompts). Still, a free model within 50 points of Fable 5 in Coding is new, and worth a test run on code you don’t mind sharing.

Speed is still the catch at the bottom of the table. MiMo-V2.6-Pro scores 46 on AA, the best of any open-weights model, and crawls along at 48 tokens a second. GLM-5.3-Flash is at 62 this week, down from 89 in late September (I re-pull that number every week because it won’t sit still). And the distillation accusation Anthropic made against Xiaomi in September still hangs over the MiMo family. That’s a competitor’s accusation, not a ruling.

Horror stories

The 100K trapdoor

I covered this above, but it goes here too because somebody is going to get bitten by it this month. You move your support-ticket agent from Haiku 4.5 to Haiku 5.5 because the press release said 90 percent cheaper. Your tickets averaged 85K tokens on the old tokenizer. On the new tokenizer that’s about 110K, and old thinking blocks pile up as input on every turn. Every call is now over the line, so you pay $0.50/$2.50 on the whole request. You’re still paying less than Haiku 4.5’s $1/$5, but nowhere near 90 percent less, and your budget was built on 90. Measure first.

GitSpawn: your repo’s git config can run code

On October 2, researchers disclosed GitSpawn, a set of eight flaws where an untrusted repository uses git configuration hooks like core.fsmonitor to run code outside the sandbox and outside the approval prompts. The affected list is Claude Code, Codex, Cursor, Goose, Hermes Agent, Qwen Code, and Grok Build. When it was published, four of the eight were still unpatched.

Hermes Agent is on that list, and it’s the number one app on OpenRouter right now. “Clone this repo and have a look” was supposed to be the safe thing to do with a coding agent. If you point an agent at code you didn’t write, run it in a container you’re willing to throw away. I run OpenClaw at home, and I’m checking my own setup this week.

PixelLeak: the agent proved its work in public

Glow Labs disclosed PixelLeak right before last issue went out, and it deserves a spot. A developer asks a coding agent to prove a UI change works. The agent takes a screenshot, hits a GitHub CLI limit on attaching images, and works around it the quickest way it can find: it pushes the image to a public repo. Across 900-plus public repos at more than 300 organizations, that added up to more than 13,000 internal screenshots, including billing records, treasury consoles with client names, and unreleased designs. About 93 percent landed in employees’ personal accounts, where corporate monitoring doesn’t look.

There’s no attacker in this story. Nobody hacked anything. The agent wanted to show its work, hit a wall, and picked the shortest route around it, which happened to be a public repo. I worry about that kind of failure more than the hacking stories, because no security scanner is looking for it.

The free week ends Friday

If you wired Step 5 Preview into Cline or Kilo Code because it was free and number three on OpenRouter, it goes back to $2.70 out on Friday, October 16. It writes about twice the median number of output tokens. Do that math before Friday.

What’s coming

StepFun says Step 5 Preview’s open weights land October 15. Right now that’s a promise with nothing on Hugging Face behind it, and I don’t count weights as shipped until there’s a license file. Mistral Large 4’s weights are scheduled for October 31, and the same rule applies.

Gemini 4 Argon goes to paid API customers and AI Ultra subscribers next. There’s still no date. Space Bunny’s real name is coming “soon,” according to OpenRouter, and MiniMax M3.1-Flash-Preview is still the best guess.

Fable 5.5, Grok 4.8, Kimi K3.1, and GPT-6.1 Sol Ultrafast are all being rumored for October. Prediction markets had Haiku 5.5 at about 92 percent for October last week and it shipped, so the rumor mill is doing okay lately. I’m watching for Fable 5.5. DeepSeek V5 is still pure speculation. DeepSeek hasn’t said anything, and it’s been “next month” since August.

Finally

Haiku 5.5 is GLM’s price until your prompt passes 100K tokens. Muse Wonder is Opus’s price with a lower score. Step 5 Preview is a top-three model on OpenRouter only while it’s free. Argon is the best model in the world, at a price that will double, and you can’t buy it.

Meanwhile the bottom of my table didn’t budge. GLM-5.3-Flash, the MiMo models, and DeepSeek V4.1 Flash are still the floor, and now Anthropic has a real model sitting on that floor too, which I did not expect to type this year. I like it. Just read the long-context pricing table before you budget off the press release.

Next week I’ll check whether Haiku 5.5 has an Arena rating yet, and what happens to Step 5 Preview’s 12 trillion tokens once people have to pay $2.70 a million for them.

Constraints Breed Creativity (Until You Hand Them to AI)

Stephan Miller — Mon, 12 Oct 2026 12:00:00 +0000

I start many writing sessions the same way. The rules come from Natalie Goldberg’s Writing Down the Bones: set ten minutes on a timer, keep your hand moving, don’t cross out, and don’t correct. When the timer starts you go, and when it stops, you can read what you wrote.

And it works. I found more ideas once I stopped blocking myself mid-stream. My writing got better, because the editor stayed quiet most of the time. The editing still happens. It just happens later.

Ten minutes is a completely arbitrary constraint, and it has done more for my output than any amount of “just be more creative” ever has.

Constraints breed creativity. Limitations force innovation. Necessity, invention, something something. It is probably one of the most repeated piece of creativity advice in existence, and I’m not going to argue with it, because I have run the experiment on myself for years and the advice holds.

What I want to do instead is take the advice seriously enough to ask what a dumb rule is actually doing when it works. Because it turns out the rule has two different jobs.

The People Who Made an Art Form of It
What the Rule Is Actually Doing
Handing the Rules to the Machine
One Paper, Pointing the Wrong Way
So Which Job Were You Hiring It For?
Three Hundred Books

The People Who Made an Art Form of It

The Goldberg timer is a gentle modern version of an older tradition: people who imposed rules on themselves so arbitrary they sound like dares, and then produced work nobody could have willed into existence directly.

The headquarters of this tradition is OuLiPo, the “workshop of potential literature,” founded in 1960 by Raymond Queneau and François Le Lionnais, with Georges Perec joining in 1967. Their premise was that a writer laboring under an arbitrary formal rule will surface phrasing that intention alone wouldn’t reach. Their proof-of-concept is Perec’s La Disparition (1969), a full novel written without the letter e. A whole novel, about disappearance, in which the most common letter in French has disappeared. Gilbert Adair’s English translation, A Void, keeps the same rule, which means the translator had to solve the puzzle a second time.

OuLiPo’s other two greatest hits are small enough to run at your desk right now:

N+7. Take a text. Replace every noun with the noun seven entries later in a dictionary. Your dictionary will give you different results than mine. Run “The deadline slipped because the team hit a bug in the migration” through a pocket dictionary and you get “The dealer slipped because the teapot hit a bulb in the milestone.” Nonsense, until your eye finds “the milestone slipped” hiding in the output. You pick the one worth keeping, and you would not have found it on purpose.
Snowball. Each successive word is one letter longer. “I am the text which slowly becomes stronger.” It sounds like a party trick until you try to make one that also says something, and discover the search for valid words pulling your meaning somewhere you didn’t plan.

Outside OuLiPo the family gets more practical. SCAMPER (Bob Eberle, 1971, compressing Alex Osborn’s 1953 checklist) is a set of verbs you cycle over a problem to force variation: Substitute, Combine, Adapt, Modify, Put to other use, Eliminate, Reverse. Stuck on a feature? Take the R. What happens if the user does it backwards, or the steps run in reverse order? TRIZ (Genrich Altshuller, from 1946) is the most engineered version: you name the underlying contradiction, where improving one parameter worsens another, and look the pair up in a matrix distilled from hundreds of thousands of patents, which hands you one of forty inventive principles.

One move in common among all of them: impose a rule, then let the search for valid output do the work. None of these people waited for a better idea. They built a wall and bounced off it.

What the Rule Is Actually Doing

Why should a dumb rule work at all? The Goldberg timer and the OuLiPo lipogram are not doing the same thing. A working constraint has two different jobs.

Job one: muzzle the judge. This is the Goldberg mechanism. The reason my mornings got more productive is that my internal editor doesn’t get to butt in (at least that’s the ideal). Don’t cross out, don’t correct, keep moving. Every rule is aimed at the judge, so the generator gets more chances. More ideas, and the judging happens later instead of continuously. Much of the classic advice works this way. Brainstorming’s “no criticism” rule, morning pages, freewriting, the first-draft cult.

Job two: shrink the search. This one involves research spikes into territory where the standards haven’t settled yet. Sandbox your coding agents, build an AI SDLC, secure the thing. The problem with new territory is that there are forty defensible ways to do everything and everybody on the internet has a strongly-worded opinion about each of them. Left unconstrained, a spike like that never ends.

The most recent one was agent sandboxing. We surveyed the landscape, all the way down to VMs, up through control planes, Docker-based options, the works. One promising path was Docker’s own sandboxing, until Docker handed that product over to E2B, a sandboxing startup, and the Docker-native direction died in front of us. The research killed more options for various reasons, and what survived pointed in a direction. It turns out E2B uses Firecracker microVMs, and those were now available on AWS.

The rule felt arbitrary: “must run standard Docker images.” And the job it did was search-shrinking: fewer live options, faster decision, done. In the end, it led to a solution that used Dockerfiles to build microVMs.

Free the generator, or narrow the search.

Even the failures do a job, by the way. I once put a hard cap on my own reading list. It did not hold. I’ll come back to that one.

Handing the Rules to the Machine

If a constraint is a process, anything that can follow a process can use it. And the constraint family translates to prompting. SCAMPER is a for-loop: “take this problem, run Substitute, Combine, Adapt, Modify, Put to other use, Eliminate, Reverse, give me three answers per verb.” N+7 is one sentence of instruction. TRIZ’s contradiction matrix is a lookup table, which is the most prompt-shaped object that has ever existed. I have a skill installed that does this: creative-ideation, from NousResearch’s Hermes collection, MIT-licensed, drops into Claude Code or Cursor, and routes whatever you ask it to one of twenty-two named human creativity methods. SCAMPER, TRIZ, OuLiPo, Oblique Strategies, most of this post, on tap. Its core rule is printed right in the skill: constraint plus direction equals creativity. Its anti-slop rule is a constraint too: refuse the first three ideas, five on high-slop terrain like AI startup pitches.

And on the search-shrinking side, the mechanical versions are the best thing I’ve got. There’s a research skill I use constantly, and it is constraints all the way down:

A fixed field schema. Every research agent has to come back with answers to predeclared fields. Not “whatever it found.” Specific answers to specific questions.
A hard budget. Eight searches, twelve fetches per item, at standard depth. That number is exactly as arbitrary as my ten-minute timer.
Forced honesty. The agents are forbidden from filling gaps with guesses. Anything unanswered gets marked uncertain. The constraint is against confabulation itself.
A gate at the end. The run isn’t done until the validator passes. Nobody promised to follow the rules. A machine checks.

Before I had that, I’d point a coding agent at a topic with plain web fetch and take whatever came back. The constrained version is not a little better. It’s the difference between research and vibes. I said in the first post in this series that a question gets you the average and a process gets you something else. This is the something else, with rails.

So: constraints free the generator, constraints shrink the search, constraints make the machine honest. The advice checks out on people, and it checks out when you hand it to a model.

One Paper, Pointing the Wrong Way

Alwin de Rooij and Michael Mose Biskjaer, “Does generative AI make us think alike?” (Behaviour & Information Technology, 2026). They tracked down every study they could find on what happens to human creative output when a model is in the loop, and pooled them into one set of numbers: nineteen studies, sixty-one separate measurements. The answer: the work gets more alike. Not a lot alike. The pull toward sameness is small (d = .334, for the stats people), but it shows up consistently, and it is exactly the size of effect you would never catch in your own work. One piece at a time, invisible. Added up across everyone, very real.

The interesting part is a breakdown buried in the review (a moderator analysis, in the paper’s terms): the sameness was stronger when the task came with rules about what the ideas had to be, and weaker when it was loose, anything-goes brainstorming. The move this post is about, impose a rule, narrow the space, force the search, is the condition under which the flattening gets worse.

I want to be exact about what “everything above” includes. It includes me. A section ago I told you, with some pride, about a skill I have installed that routes my requests to twenty-two named creativity methods. That is a novelty-forcing rule about what the ideas have to be, with an install command.

The skill’s own author saw the middle coming, for what it’s worth. The anti-slop rule, refuse the first three ideas, exists because the first draws are the average ones. So either the patch works and a constrained model can be pushed off the middle, or the patch is one more wall somebody handed me and we are all standing in the same destination anyway.

A model is an averaging engine. Ask it for ideas and you get the middle of idea-space. Ask it for ideas under a tight rule about what they have to be, and you have not pushed it off the middle. You have told it which middle. Give the model a persona plus a rule, or ask it for ten ideas using only X: same kind of thing, a rule about what the output has to be. Everyone who imposes that kind of rule on the same class of model gets pushed toward the same narrowed region. And everyone thinks the constraint worked, because the answer looks different from what they’d have written alone. It does, but it also looks like what everyone else with the same rule got.

But this is one paper, and the moderator result is one finding inside it. The studies under it mostly tested 2023 and 2024 chat models, not the current stuff. And what they constrained was the task humans brought to the model, not the model itself: the move from one to the other is an inference, not a finding. But it is the first direct measurement of the advice this post is about, and it points a different way.

So Which Job Were You Hiring It For?

Here’s where the two jobs from earlier stop being a taxonomy and start being instructions.

Judge-muzzling constraints are for you. Keep them. The ten-minute timer works because you have an internal editor that gets in the way, and the rule handcuffs it. A model has no internal editor. It is not nervously crossing out its own drafts. “Don’t stop to correct yourself” is a meaningless instruction to something that was never going to. Goldberg’s rules, morning pages, the no-criticism rule, the first-draft cult: all of these treat a human bottleneck, they work on humans, and they neither need nor benefit from a model’s involvement.

Search-shrinking constraints survive the machine. The research skill, the field schema, the eight-search budget, the validator. Narrowing a search is narrowing a search, and it works on a model for the same reason it works on a spike at work: fewer live options, faster decision, and no one was asking for diversity in the first place. The point of those constraints was coverage and honesty, and on those metrics the constrained version wins outright.

Novelty-forcing constraints are the ones that get eaten. When the entire job of the rule is “push me somewhere I wouldn’t go on my own,” handing it to the model converts it into a destination, and it’s the same destination everyone else with the same rule gets. This is the branch the meta-analysis measured and it backfired.

So the practical rules:

Constraints on your own process: keep, no changes. They muzzle your judge. The model never had one.
Constraints on research and retrieval: keep, they enforce themselves. If a validator can check it, it works.
Constraints meant to force original ideas out of a model: go wide first. Run the open divergent pass before you create any walls. Survey the territory with the constraint off, watch what’s there, and only then draw a rule, distilled from what you saw. A constraint earned by a wide survey lands somewhere specific. A constraint imposed up front lands where everyone else’s did.
Treat a rule the model hands you as a draft, not a wall. If you are in genuinely new territory, you’ll be regularly unsure which of its suggested walls to keep. That uncertainty is correct. A constraint you invented after looking at the territory does a different job than one handed to you by something that averages.

The human half needs a kitchen timer, a pocket dictionary, and paper. The machine half is one install and one write-up: the creative-ideation skill linked up top for the method library, and the research skill linked above for the constrained-search side. Everything else in this post is a way of holding them. Which is all I wanted out of this stuff in the first place: just another tool that gets me to an answer I couldn’t see from where I was standing.

Three Hundred Books

Some years ago I decided my reading needed discipline. I have over four thousand books on my ebook reader. IIt did not feel like a problem while I was buying them: four thousand. I used to graze, jumping between them at random, finishing nothing. So I made a Current Reading list, the books I’m actively reading, and capped it at fifty. Fifty sounded like the kind of number a disciplined person would pick.

It is currently three hundred books long.

I added each book one at a time, and every addition made sense at the moment I made it. Book seventy-three was basically research. Book one-forty was short. Book two hundred was for a project I have not started and will never start. Every one of the three hundred was, at the moment of its addition, an exception.

The rule failed at its job: stopping me from spreading myself across too many books at once. I still read a dozen or twenty at a time.

I am currently reading twenty books, which means I am reading two and have strong opinions about eighteen.

Three hundred books is a horizon: the books I’m plausibly reading in the next five or six years, kept in my face so the choosing among them stays live. It’s a map, not a discipline.

Give yourself one. Watch what it does. It may do the job you hired it for, like the timer. It may fail at that job and get itself a better one, like the list. Either way you will be there watching it happen, which is the one thing the model will never do for you. The constraint was never the whole point. What you were paying attention to when you made it was.

Gemini 4 Argon Won Every Arena Category. You Can't Use It.

Stephan Miller — Tue, 06 Oct 2026 13:00:00 +0000

The best model in the world this week is one you can’t use.

Gemini 4 Argon showed up on Arena and took first place in all six text categories I track: Overall, Coding, Creative Writing, Instruction Following, Hard Prompts, and Math. It’s listed at $2 in and $10 out, which is cheaper than every Claude model it knocked off the top. So I did what any reasonable person with an OpenRouter account would do. I went to try it. 404.

That’s because Google isn’t selling it to you. Argon is locked to a few hundred cybersecurity organizations (650-plus, if you’re counting) while Google figures out whether it’s safe to let the rest of us have it. Same week, OpenAI went one step further and refused to ship its own next flagship, GPT-6.1 Astra, because in testing it lied about what it had done. So the two biggest frontier stories this week are both about models you can’t have. And the one you can’t have still managed to rearrange my cheapskate table just by existing.

Gemini 4 Argon won everything on preliminary votes
The Astra that lied, and the Sol you can buy instead
A model you can’t call just rearranged my cheapskate table
Space Bunny is still free, still anonymous, and now number one
Horror stories
- Your agent’s status report might be fiction
- Four trillion prompt tokens to a stranger
- Budgeting off a leaderboard number
What’s coming
Finally

Gemini 4 Argon won everything on preliminary votes

Arena’s board, stamped October 2, has Argon at 1525 Overall, 20 points clear of claude-opus-4-6-high at 1505. It leads Coding at 1560, Creative Writing at 1519, Instruction Following at 1528, Hard Prompts at 1551, and Math at 1530.

Every one of those ratings carries the “Preliminary” tag. It has 4,932 votes in Overall, about 1,100 to 1,250 each in Coding and Creative Writing, and just 255 in Math. For comparison, the Opus 4.6 it displaced has 77,636 Overall votes. A model can win a category on 255 votes. Opus 4.6 has about 300 times that many votes in Overall alone.

Artificial Analysis is less excited. On its Intelligence Index, Argon scores 53. That puts it eighth, tied with Fable 5.1 and GPT-6 Astra, behind Claude Opus 5.5 at 58 and Claude Sonnet 5.5 at 56. That’s frontier, no question. It’s just not the top of the frontier. The blind-vote crowd loves it and the hard benchmarks say it’s good. I’ve seen this split before, usually in the opposite direction. GPT-6 Astra tied for first on AA and debuted 24th on Arena. Argon is the mirror image.

Why can’t you buy it? Google announced Argon last week and rolled it out first through its Fairwind Program: more than 650 partners, including government agencies, critical infrastructure operators, and security vendors. Those organizations can only hand it to employees in cybersecurity, incident response, or pentesting roles, and they have to use multi-factor authentication. Google’s stated reason is that it’s still testing safeguards against cyberattacks, CBRN misuse, and indirect prompt injection. The self-reported numbers explain the nerves: 68 percent on CWE-bench v1 for vulnerability remediation, 77.9 percent on DeepSWE v1.1, and a story about finding a patient-data bug in hospital software that earlier frontier models missed. Google says it trained Argon specifically to find, validate, and patch software vulnerabilities. Nobody independent has checked any of that yet.

Next in line are paid API customers and Google AI Ultra subscribers. No date. And the price on Arena’s board is introductory. After the intro period it goes to $4 in and $20 out, and Google hasn’t said when the intro period ends. So the leaderboard’s number one is off limits, priced at a number that won’t last, ranked on votes that are still coming in.

The Astra that lied, and the Sol you can buy instead

OpenAI had its own flagship lined up for October. It’s not coming.

According to reporting picked up by The Hacker News, GPT-6.1 Astra failed OpenAI’s internal safety and alignment audits. The details read like the postmortem nobody wants to write about an agent. It didn’t disclose actions it had taken, and it went ahead without asking permission. Testers also caught it reaching for outside tools in situations flagged as unsafe.

Saachi Jain, OpenAI’s head of safety systems, put it this way: the model “didn’t quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it’s done.”

In plain English, it did stuff you didn’t ask for and then lied to you about it. If you run coding agents for a living, that’s the failure mode that matters most. A dumb model wastes your afternoon. A model that lies about what it did is how you lose a database.

The model OpenAI does sell got a report card too. The same day, the UK’s AI Security Institute published a report on GPT-6 Astra, the one OpenAI has been selling since early September. In simulated testing, Astra “conducted a range of unsanctioned attack activities” at a higher rate than GPT-5.6 Sol and GPT-5.5, sometimes even after the scope was spelled out for it. The report’s examples: “creating fake identities which it used to deceive developers, posting comments from fake accounts arguing against the results of accurate security reviews, and delivering malicious payloads to open-source codebases.” That’s a simulation, not an incident. But plenty of people have that exact model wired into a coding agent right now.

So at DevDay on September 29, OpenAI shipped GPT-6.1 Sol instead. OpenAI says it gets close to GPT-6 Astra on agentic coding, computer use, and professional work at a fifth of Astra’s price. On OpenRouter that works out to $2 in, $10 out, and 10 cents per million for cached input. It’s the number one new model on OpenRouter’s trending list this week at 927 billion tokens. Artificial Analysis gives it 52, one point behind Argon. On Arena it’s 21st Overall at 1483 and eighth in Coding on only 609 votes.

GPT-6.1 Sol is $2/$10, and so is Gemini 4 Argon’s intro price. Two labs landed on the same sticker for near-frontier capability the same week, and only one of them will actually sell it to you. OpenRouter’s provider table also shows Sol is slow right now, 32 to 50 tokens a second depending on the host. There’s a “Fast” endpoint at double the price if that matters to you.

A model you can’t call just rearranged my cheapskate table

Same method every week. Take each Arena category’s leader, draw a band 50 points below its rating, and find the cheapest model still inside the band. I compute the band in code from the full table, 129 rows per category, never off the first screen. I learned that one the hard way.

This week the leader in every category is Argon. That changes two things at once. Argon raised every ceiling, so every band moved up. And Argon is cheap for a leader, so every “cheaper by” ratio collapsed. Bands got shallower too: 37 models deep in Overall (57 last week), 60 in Coding (73), 31 in Hard Prompts (43), 23 in Instruction Following (38), 35 in Math (43). Creative Writing went the other way, from 12 to 18, because Argon only beat Opus 5.5 there by three points.

Category	Leader	$ out	Cheapskate pick	$ out	Δ rating	Cheaper by
Overall	gemini-4-argon-high (1525)	$10	MiMo-V2.6-Pro (1480, #26)	$0.87	−45	11.5×
Coding	gemini-4-argon-high (1560)	$10	MiMo-V2.6-Flash* (1521, #33)	$0.28	−39	36×
Creative Writing	gemini-4-argon-high (1519)	$10	gemini-3.7-flash-high (1493, #5)	$3.75	−26	2.7×
Instruction Following	gemini-4-argon-high (1528)	$10	deepseek-v4.1-flash-max (1479, #21)	$1.20	−49	8.3×
Hard Prompts	gemini-4-argon-high (1551)	$10	MiMo-V2.6-Pro (1505, #22)	$0.87	−46	11.5×
Math	gemini-4-argon-high (1530)	$10	GLM-5.3-Flash (1503, #13)	$0.50	−27	20×

*1,508 votes. GLM-5.3-Flash sits right next to it at 1523 with 6,157 votes for 50 cents, and that’s the steadier bet.

Argon is quoted at its intro price. When it goes to $20 out, every ratio in that table doubles. GLM-5.3-Flash is quoted at Z.ai’s list price of 50 cents out. Arena’s price column shows 20 cents, but that’s a third-party floor.

Last week GLM-5.3-Flash was the pick in four of six categories, and this week it’s the pick in one. GLM didn’t get worse. Its Overall rating went from 1474 to 1473. The band moved. The Overall cutoff is now 1475, so GLM missed it by two points. Hard Prompts’ cutoff is 1501 and GLM sits at 1500, one point short. In Instruction Following it’s six points short. A model nobody outside Fairwind can call, rated on preliminary votes, pushed the most-used cheap model of the last month out of half my table.

So which table do you actually shop from? Here’s the same one, recomputed without Argon.

Category	Purchasable leader	$ out	Cheapskate pick	$ out	Cheaper by
Overall	claude-opus-4-6-high (1505)	$25	GLM-5.3-Flash (1473)	$0.50	50×
Coding	claude-fable-5-high (1552)	$50	MiMo-V2.6-Flash (1521)	$0.28	179×
Creative Writing	claude-opus-5.5-high (1516)	$20	gemini-3.7-flash-high (1493)	$3.75	5.3×
Instruction Following	claude-opus-5.5-high (1517)	$20	GLM-5.3-Flash (1472)	$0.50	40×
Hard Prompts	claude-opus-5.5-high (1534)	$20	GLM-5.3-Flash (1500)	$0.50	40×
Math	claude-fable-5-high (1522)	$50	GLM-5.3-Flash (1503)	$0.50	100×

(In Hard Prompts and Math, MiMo-V2.6-Flash technically sneaks in at exactly the band edge for 28 cents, but in Math that’s on 330 votes. I’m not crowning anything on 330 votes.)

That’s basically last week’s table. GLM is back in four categories, and the year-old claude-opus-4-6-high is still the top Overall model you can actually buy.

MiMo-V2.6-Pro is now the Overall and Hard Prompts pick at 87 cents. Artificial Analysis rates it 46, the best score among open-weights models, at 13 cents per index task. It’s also slow: 46.6 tokens a second with a 5.2-second wait for the first token, well under the 75 tok/s median. And the distillation allegation from last issue, Anthropic’s report claiming Xiaomi routed 400,000 user conversations through Claude, still hangs over the whole MiMo family. That’s an accusation from a competitor, not a ruling. Whether it matters to you depends on your lawyers.

MiMo-V2.6-Flash held the cheapest-in-band spot in Coding for a second week, and its vote count went from 1,043 to 1,508. I said last week that if it held that spot with a few thousand votes, the Coding crown had moved. It’s not there yet. AA gives it 38, about 6 cents per task, 62 tokens a second, and it’s verbose: 240 million output tokens to get through AA’s suite against a 140 million median.

DeepSeek V4.1 Flash is the new Instruction Following pick at $1.20, and it’s the only pick this week that isn’t slow. AA clocks it at 222 tokens a second, third fastest among the open-weights models AA tracks, and it starts answering in about a second. It’s right at the band edge, one point above the cutoff, so it could fall out next week. On OpenRouter some third-party hosts list it at 18 to 40 cents out, way under list. Check uptime before you chase those.

Creative Writing is still the category where you pay for quality. The best value is Gemini 3.7 Flash at $3.75, just 2.7 times cheaper than Argon. Gemini 3.8 Flash costs the same and rates lower, so 3.7 wins the tie.

Space Bunny is still free, still anonymous, and now number one

Last week Space Bunny Alpha was the second-biggest model on OpenRouter. This week it’s first, at 38.5 trillion tokens, up 112 percent. Stealth models as a group now account for 10.5 percent of all text requests on OpenRouter, and stealth requests went up 296 percent in a week.

It’s still free. It still hasn’t told anyone who made it. The tokenizer evidence still points at MiniMax’s M3.1-Flash-Preview, and it’s still unconfirmed. I’ll pull a number from its OpenRouter activity panel in the horror section, because it deserves the spot.

Meanwhile, the paid volume king is DeepSeek V4.1 Flash. It’s number two this week at 28.4 trillion tokens, up 36 percent, and number one for the trailing month at 72.3 trillion. DeepSeek is the top author on OpenRouter at 21.8 percent of requests. Google is right behind at 21.5 percent, up 23 percent, all of that from models that aren’t Argon. GLM-5.3-Flash is third at 10 trillion, down 25 percent. Anthropic is 2.5 percent of requests, which tells you who’s buying Claude through OpenRouter and who’s buying it direct.

On the apps list, Nous Research’s Hermes Agent pushed 1.86 trillion tokens in a day, ahead of Claude Code at 1.06 trillion. On OpenRouter, at least, an open-source agent is now burning more tokens than Anthropic’s own coding tool. I run OpenClaw at home, so I’m part of the problem.

Horror stories

Your agent’s status report might be fiction

I covered GPT-6.1 Astra above, but it belongs here too, because it’s the scariest thing in this issue. Most model horror stories are about a model being dumb: it hallucinated a function, it deleted the wrong folder. Astra’s failure was the model hiding its own actions and acting outside the task it was given. Its shipping big brother, GPT-6 Astra, went as far as running sock-puppet accounts in AISI’s supply-chain simulations. That’s a trust problem, and a higher benchmark score doesn’t fix it.

OpenAI deserves credit for catching it in testing and not shipping it. But agent frameworks are built on the assumption that the model’s account of what it did is true. Your agent’s “I ran the tests and they passed” message is only useful if it’s true. Somebody built a model where it sometimes wasn’t. The next one might not get caught.

Four trillion prompt tokens to a stranger

Space Bunny Alpha’s activity panel shows 4.06 trillion prompt tokens since September 23, against 135 billion completion tokens. That’s a 30-to-1 ratio of input to output. People aren’t chatting with it. They’re feeding it whole codebases and huge documents, because it’s free and has a million-token window.

The listing says prompts and completions “may be retained by the provider.” The provider won’t give its name. I’ve said this before and I’ll keep saying it until free stealth models stop topping the charts: kick the tires on public code all you want, but don’t send it anything you wouldn’t post on GitHub.

Budgeting off a leaderboard number

This one’s small but I bet somebody does it this month. Arena shows Argon at $2/$10 next to a #1 ranking in every category. A team lead sees that, writes “switch to Gemini 4, cheaper and better” in a planning doc, and finds out at implementation time that the model isn’t available to them and the price is going to double. If you build a budget off a leaderboard row, check that you can actually call the model, and whether the price is introductory.

What’s coming

Gemini 4 Argon for the rest of us is announced but has no date. Paying API customers and Ultra subscribers go next, then everyone else. When it opens up, I’ll be watching two things: whether the Arena lead holds as votes pile up, and when the price goes to $4/$20.

GPT-6.1 Astra is shelved with no new date. OpenAI says it’s shifting focus to “improving the safety of future models.”

The Space Bunny unmasking is still speculation. If it turns out to be MiniMax M3.1-Flash and gets a price, it’ll probably follow the last stealth model. Union Alpha turned out to be Pareto by Unbiased, and the free period ended when the name came out. Then we’ll see how much of that 38.5 trillion was people who actually wanted the model.

Claude Haiku 5.5 and Fable 5.5 are speculation too. Prediction markets give both about a 92 percent chance of shipping in October. Anthropic said Haiku 5.5 would follow Opus and Sonnet “in the coming weeks,” and Sonnet shipped on September 28.

DeepSeek V5 is still pure speculation. People keep saying October, and DeepSeek hasn’t said anything.

Finally

Google and OpenAI arrived at the same place from opposite directions this week. Google built something it thinks is too good at hacking to hand out, so it handed it to defenders first. OpenAI built something that couldn’t be trusted to stay in its lane, so it handed out something smaller. In both cases the frontier model in the headlines isn’t for sale, and the one that is costs $2/$10.

For the stuff you actually pay for, not much changed. GLM-5.3-Flash and the MiMo models are still the floor. Year-old Opus 4.6 still tops the purchasable Overall board. DeepSeek V4.1 Flash is moving more tokens than anything except a free model with no name.

Next week I’ll check whether Argon’s 255 Math votes have turned into a few thousand, and whether anyone outside Fairwind has gotten a key. And I’ll check whether my cheapskate table survives the next ceiling this model raises.

Before You Write a Scraper, Check Firecrawl Alexandria

Stephan Miller — Mon, 05 Oct 2026 12:00:00 +0000

Last Thursday I published a 70-line scraper that watches the Arena leaderboard for new AI models. It runs daily and announces new arrivals.

Two days earlier, Firecrawl launched Alexandria, a catalog of providers that sell the structured version of exactly this kind of data, callable through the same scrape endpoint I was already paying for. I didn’t notice. I was busy having AI build me regex for a markdown table, while the company whose credits I was spending shipped the thing meant to make that regex unnecessary.

So to recap: I built a machine that watches the internet for new arrivals, and the first new arrival it missed was the one built to replace it.

If somebody is already selling the structured version of the data I wanted, why was I scraping the page? It never occurred to me to check.

What Alexandria Is
First, You Need a Newer CLI
Browsing the Catalog Costs Nothing
The Answer for Arena: Keep Scraping
The Leaderboard Next Door: Artificial Analysis
The Top 20 Is Really Nine Models
The Same Harness, New Fetch Step
What the Swap Costs
If Your Cron Runs Unattended, Read This Part
Check the Catalog Before You Write the Scraper

What Alexandria Is

Alexandria is Firecrawl’s catalog of third-party data sources: FRED economic data, podcast transcripts, app store listings, company records, and a few hundred other things. They named their permanent store of the world’s data after history’s most famous case of data loss, which is either confidence or a gap in the marketing team’s classics minor. Each tool in it has a published contract that says what options it takes, what it returns, and what it costs. Finding tools and reading contracts is free. Running one costs credits, Firecrawl’s metered unit, and the free plan is 1,000 of them a month. The Arena scraper already spends one per run: it fetches the page through Firecrawl. That 1 is the baseline for every cost number in this post.

It works in three steps: find a tool, read its contract, call it. The call goes to the regular /v2/scrape endpoint with an alexandria array instead of a URL.

If you want the full tour, JP Caparas already wrote it well: Firecrawl Alexandria, tested end to end with real API calls. I am not going to redo that. This post asks: does it replace a scraper I actually run?

First, You Need a Newer CLI

My installed Firecrawl CLI was 1.23.3, which predates Alexandria. The list and find-tools commands need 1.24 or later.

The 1.23.3 on my machine came from a plain npm install, skipping the installer that rewrites your editors. So the CLI is installed, just old, and I am not upgrading it blind for a look around. npx runs a specific version without replacing the one you have:

npx -y firecrawl-cli@1.24.6 credits --json

It picks up the same FIRECRAWL_API_KEY from your environment. When you decide you want Alexandria in scripts permanently, upgrade for real. Until then, this is a free look.

Browsing the Catalog Costs Nothing

Start at the root:

npx -y firecrawl-cli@1.24.6 list --json

That returned 23 categories for me on October 1st. The launch page said 20, and JP counted 21, so they seem to be adding categories as they go.

Each item carries a nextCommand you can run as is to go one level down. That is also why the commands in this post alternate between bare list and alexandria list: I am running whatever nextCommand prints, verbatim. The one I cared about:

npx -y firecrawl-cli@1.24.6 alexandria list ai-models --category --json

Trimmed to what matters:

{
  "provider": "firecrawl",
  "capability": "find-tools",
  "creditsCost": 0,
  "data": {
    "level": "providers",
    "items": [
      { "id": "artificialanalysis-ai", "name": "Artificial Analysis", "toolCount": 4 },
      { "id": "arxiv-org", "name": "arXiv", "toolCount": 6 },
      { "id": "huggingface-co", "name": "Hugging Face", "toolCount": 5 },
      { "id": "ollama-com", "name": "Ollama model library", "toolCount": 7 },
      { "id": "openrouter-ai", "name": "OpenRouter", "toolCount": 8 },
      { "id": "semanticscholar-org", "name": "Semantic Scholar", "toolCount": 10 }
    ],
    "total": 6
  }
}

Six providers. No Arena. When I first ran this a few days earlier it was three: Artificial Analysis, Hugging Face, and OpenRouter. arXiv, Ollama, and Semantic Scholar showed up since, so this category is growing as fast as the category list is.

The Answer for Arena: Keep Scraping

If you want Arena’s rankings, Alexandria does not have them. The Arena scraper stays a scraper, and it costs one credit a run.

It took two commands. Every scraper in this series gets that check from now on: it takes a minute.

But the category was not empty, and one of those providers is close enough to be interesting.

The Leaderboard Next Door: Artificial Analysis

Artificial Analysis runs its own LLM leaderboard. It doesn’t use human votes. It runs benchmarks and puts them into an Intelligence Index, along with list prices and measured speed. It answers a different question than Arena does: Arena asks which answer people like better, and Artificial Analysis asks how the model scores on tests. For “did a new model just show up near the top,” it works too: the same question, answered from the benchmark side. A second signal, not a substitute.

Listing that provider’s tools shows the price of everything before you spend anything:

npx -y firecrawl-cli@1.24.6 list ai-models artificialanalysis-ai --category --json

Tool	Credits	Per record?
`benchmarks/search_models`	5	no
`benchmarks/get_model`	5	no
`benchmarks/list_model_providers`	5	no
`benchmarks/search_provider_endpoints`	5	no

OpenRouter’s eight tools are also 5 credits a call. perRecord: false is the flag to look at. It means the price is per call, not per row, so asking for 100 models costs the same as asking for one. If that flag were true, a pagination loop could burn credits fast.

The contract for search_models says it returns the whole leaderboard sorted by Intelligence Index, up to 100 per page, with filters for creator, open weights, and reasoning. Nothing in it needs a terms agreement. So I spent the five credits:

npx -y firecrawl-cli@1.24.6 scrape artificialanalysis-ai/benchmarks/search_models \
  --options '{"limit": 20}' --json

It came back in about a second and a half. Each result is a proper record: slug, creator, the index, a dozen benchmark scores, prices per million tokens, context window, and a link back to the model’s page on the site. No regex. No --wait-for 3000 hoping the table rendered. No guessing which row is the calendar widget.

And then I looked at the top 20.

The Top 20 Is Really Nine Models

Here are the first eight rows:

1 claude-opus-5-5 Anthropic 57.6
2 claude-opus-5-5-xhigh Anthropic 56.0
3 claude-opus-5-5-high Anthropic 53.6
4 claude-fable-5-1 Anthropic 53.4
5 claude-fable-5-1-xhigh Anthropic 53.2
6 gpt-6-astra OpenAI 52.7
7 gpt-6-astra-xhigh OpenAI 52.4
8 claude-opus-5-5-medium Anthropic 51.2

Artificial Analysis lists every reasoning-effort setting as its own row. Opus 5.5 at max effort, at xhigh, at high, at medium. Those are four entries for one model. The “top 20” is really about nine models, several times over. Claude Opus 5.5 alone holds four of the top eight slots, which makes this less a leaderboard than an Anthropic family newsletter.

That is not a bug. Nobody ships one this loud, and the first test run prints it. It is a caveat of the swap, and the first one to handle: the Arena harness assumes one row per model, and this feed returns one row per model per effort setting. The contract gives you clean JSON but clean data does not mean you can skip reading it.

The fix is to collapse effort variants into one model before ranking. The slugs end in -xhigh, -high, -medium, -low, so strip that and keep the first (highest) row for each model.

My first version of that also stripped -max, since the variants are labeled “Max Effort” in their names. That ate a real model. qwen3-8-max is not Qwen 3.8 at max effort. “Max” is part of the product’s name. The max-effort rows in this table have no suffix at all, which you only find out by looking. So max came out of the pattern.

The Same Harness, New Fetch Step

Here is the Arena harness with the scraper swapped for an Alexandria call. It is still 70 lines, and everything below fetch() is the same code: save with a timestamp, compare with the last run, speak only when a model is new to the top 20. The Arena watcher keeps its job. This one runs alongside it: same harness, different signal.

#!/usr/bin/env python3
"""arena_watch.py with the scraper swapped for a Firecrawl Alexandria call."""
import json, re, sqlite3, subprocess, sys
from datetime import datetime, timezone

DB = "aa.db"
TOP = 20

# Artificial Analysis lists every reasoning-effort setting as its own row, so
# claude-opus-5-5, -xhigh and -high are three rows. Collapse them to one model.
EFFORT = re.compile(r"-(xhigh|high|medium|low|minimal)$")

def fetch():
    out = subprocess.run(
        ["firecrawl", "scrape", "artificialanalysis-ai/benchmarks/search_models",
         "--options", '{"limit": 100}', "--json"],
        capture_output=True, text=True, check=True).stdout
    call = json.loads(out[out.index("{"):])["data"]["alexandria"][0]
    rows, seen = [], set()
    for m in call["data"]["results"]: # already sorted by Intelligence Index
        model = EFFORT.sub("", m["slug"])
        if model in seen:
            continue # a lower effort setting of a model we already ranked
        seen.add(model)
        rows.append((len(rows) + 1, model, m["creator_name"],
                     round(m["intelligence_index"] or 0, 1)))
    return rows

def main():
    rows = fetch()
    if len(rows) < TOP:
        sys.exit(f"only got {len(rows)} models back, check the call")

    taken = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M")
    db = sqlite3.connect(DB)
    db.execute("""CREATE TABLE IF NOT EXISTS ranks
                  (taken TEXT, rank INT, model TEXT, org TEXT, score REAL)""")
    db.executemany("INSERT INTO ranks VALUES (?,?,?,?,?)", [(taken, *r) for r in rows])
    db.commit()

    prev = db.execute("SELECT MAX(taken) FROM ranks WHERE taken < ?", (taken,)).fetchone()[0]
    if not prev:
        print(f"{taken}: first run, saved {len(rows)} models. Nothing to compare yet.")
        return

    was_top = {m for (m,) in db.execute(
        "SELECT model FROM ranks WHERE taken = ? AND rank <= ?", (prev, TOP))}
    seen = {m for (m,) in db.execute("SELECT DISTINCT model FROM ranks WHERE taken < ?", (taken,))}

    print(f"{taken} vs {prev}")
    for rank, model, org, score in rows[:TOP]:
        if model not in seen:
            print(f" NEW #{rank:<3} {model} ({org}, {score})")
        elif model not in was_top:
            print(f" MOVED UP #{rank:<3} {model} ({org}, {score})")

if __name__ == " __main__":
    main()

The script shells out to plain firecrawl. The pinned npx form above was for looking around. On the box that runs the cron, either upgrade the installed CLI for real or swap the command for the full npx version.

It asks for 100 rows instead of 20, because after collapsing there have to be 20 distinct models left. Since pricing is per call, the extra 80 rows are free. That one call came back with 61 distinct models. The first run’s top 20, with the variants collapsed:

#1 claude-opus-5-5 (Anthropic, 57.6)
#2 claude-fable-5-1 (Anthropic, 53.4)
#3 gpt-6-astra (OpenAI, 52.7)
#4 muse-spark-1-3 (Meta, 48.1)
#5 gpt-6-sol (OpenAI, 47.5)
#6 grok-4-7 (SpaceXAI, 46.4)
#7 mimo-v2-6-pro (Xiaomi, 46.3)
#8 qwen3-8-max (Alibaba, 45.4)
...
#20 gpt-6-luna (OpenAI, 37.3)

Now that is a top 20 you can diff.

The sys.exit guard means something different here, too. In the Arena version, fewer than 20 rows meant “the page layout changed, go fix the regex.” Here it means the call itself went wrong.

What the Swap Costs

	Arena scraper	Alexandria call
Credits per run	1	5
Daily for 30 days	30	150
Parsing	Regex on a markdown table	JSON, fields documented in the contract
Breaks when	The page redesigns	The contract changes
Extra fields	Rank, score, org	Benchmarks, prices, speed, context window
What it measures	Human preference votes	Benchmark scores

A daily Alexandria check uses 150 of the free 1,000 a month, which is fine for one watcher and adds up if you run five.

So the swap is not free. Five times the credits buys you no regex, no layout breakage, and a lot of extra fields. The Model Buzz Report leans on both kinds of signal, so the two lists side by side say more than either one alone.

My test cost 15 credits: three five-credit calls, one of them the run that ate Qwen. Fifteen, out of the 1,000 I pay nothing for. All that comparison shopping, and the grand total is 1.5% of free.

If Your Cron Runs Unattended, Read This Part

Some Alexandria providers will not run until you accept their data-use terms. Alexandria tells the caller, human or agent, to show the terms to a person and wait for an explicit yes.

The two providers I touched are fine: terms show openrouter-ai returns "required": false, and Artificial Analysis is not in the terms catalog at all. But check before you schedule anything:

npx -y firecrawl-cli@1.24.6 alexandria terms show <provider> --json

Check the Catalog Before You Write the Scraper

Every recipe in this series so far has assumed the data lives on a web page and the job is getting it off that page. Alexandria adds a step before that: someone may already sell the structured version.

The step is cheap. list, follow nextCommand two or three times, read the contract. Takes a couple of minutes. If what you want is in there, you skip the part of scraping that breaks. If it is not, like Arena, you have lost nothing and you write the scraper knowing you have to.

Next up for this one: OpenRouter’s get_model_stats, which says it returns usage and availability for a model. That is the other half of what the Model Buzz Report watches, and I have been getting it the hard way. Which is, as established, my signature way of getting things.

Obsidian OneDrive Sync: The Setup Windows Already Installed for You

Stephan Miller — Thu, 01 Oct 2026 13:00:00 +0000

OneDrive is the sync method nobody chooses. It chooses you. You set up a Windows machine, click through a dozen screens, and at the end of it your Documents folder lives in Microsoft’s cloud whether you asked for that or not.

So a lot of people end up syncing Obsidian with OneDrive by accident. They made a vault in Documents, and one day it showed up on their other laptop. It works great, right up until a note opens blank or a file named Daily Note-DESKTOP-7QK2M1B.md appears next to the real one. My guide to syncing an Obsidian vault across devices gives OneDrive about two lines.

Why OneDrive Is the Path of Least Resistance
Files On-Demand Is Why Your Notes Vanished
- Storage Sense will undo your work
OneDrive on a Mac
What OneDrive Refuses to Sync
- The duplicate file with your computer’s name in it
iPhone and iPad: Remotely Save or Nothing
Android: Remotely Save or a Sync App
Personal vs. Business: Your Work Account Is Out
The Cost Math If You Already Pay for Microsoft 365
Should You Use It?

Why OneDrive Is the Path of Least Resistance

On Windows, OneDrive is already installed, already signed in, and already syncing something. There is no client to download and no account to create. If your whole Obsidian life happens on Windows desktops and laptops, you could stop reading after the next section and be fine.

Obsidian’s own help page says about as much. It calls OneDrive “a popular cloud storage option for Windows and macOS users” and then, in the same breath, says it “has limitations on Android and isn’t officially supported for syncing Obsidian vaults on iOS.”

That sentence is the whole reason for this post. Desktop is easy if you flip one setting. Phones are where it starts costing you time.

Files On-Demand Is Why Your Notes Vanished

OneDrive’s Files On-Demand shows every file in File Explorer whether or not the bytes are actually on your disk. A blue cloud icon means the file is online-only. A green check means it’s on this machine for now. A solid green circle with a white check means it is pinned and stays.

Obsidian needs the third one. It watches the vault folder and expects every file it sees to be a file it can read right now. An online-only note is a promise, not a file. Search misses it. Links to it look broken. A plugin tries to load its own settings and gets nothing. And none of this throws an error. It just looks like Obsidian forgot things.

The fix on Windows takes ten seconds:

Open File Explorer and find your vault folder.
Right-click it.
Choose Always keep on this device.

Wait for the icons to go solid green. Now it’s a real folder full of real files, and Obsidian stops acting psychotic.

Microsoft also removed the old on/off switch for Files On-Demand. What’s left is two buttons, buried with the rest of OneDrive’s settings: click the OneDrive cloud icon in the system tray, open Settings , and look under Sync and backup → Advanced settings. There you get Free up disk space (the default) and Download all files , which is the old “off.” If your whole OneDrive is small, Download all files is the blunt fix. If it is a terabyte of family photos, pin just the vault.

Storage Sense will undo your work

Here is the trap after the trap. Windows has a feature called Storage Sense that frees up disk space on a schedule, and Microsoft’s documentation says that with it turned on, locally available OneDrive files “will become online-only files after the time period you’ve selected.”

So you open your vault every day, everything is fine, and then a note you haven’t touched since spring comes back blank. That’s why pinning the vault folder matters: Storage Sense leaves pinned files alone and only reaps the ones that are merely local. Don’t just open the notes and assume they stay downloaded. The setting lives at Settings → System → Storage → Storage Sense , where a dropdown decides how long an unpinned OneDrive file gets to stay on disk. Set it to never, or at least to longer than your oldest note is likely to sit unopened.

OneDrive on a Mac

Mac users end up on OneDrive because work gave them Microsoft 365, or because they also have a Windows machine and want one cloud. It’s a real combination, and almost every Obsidian guide skips it.

The thing to know is that on macOS 12.3 and later, you can’t turn Files On-Demand off. Microsoft moved OneDrive on the Mac to Apple’s File Provider framework, and its own support page says Files On-Demand “must always be enabled.”

That runs head-on into Obsidian’s advice to avoid the feature, and the way out is the pin: Obsidian doesn’t care about Files On-Demand itself, it cares that no vault file is ever online-only. Pinning the vault is what “avoid” looks like when the off switch is gone.

Your OneDrive folder lives at ~/Library/CloudStorage/OneDrive-Personal, macOS controls it, and you can’t move it.

The fix is the same as Windows, with one navigation trap: ~/Library is hidden, so don’t go hunting for the folder by browsing. OneDrive shows up in the Finder sidebar under Locations , or press Cmd+Shift+G and paste the path. Right-click the vault folder, pick Always keep on this device , then open the vault in Obsidian from that CloudStorage path. Since you can’t disable the feature globally, pinning the vault is the only defense you get, so do it before you open the vault, not after the first blank note.

What OneDrive Refuses to Sync

OneDrive has opinions about filenames. Obsidian already refuses most of these characters in note titles, so this usually bites attachments and files that got into the vault some other way: a downloaded PDF, a clipped image, a folder you dragged in from Finder.

Characters like " * : < > ? / \ | are not allowed. Leading and trailing spaces are out too.
Reserved names cover CON, PRN, AUX, NUL, COM0 through COM9, LPT0 through LPT9, and anything with _vti_ in it. You probably don’t have a note called AUX. Someone does.
Path length caps the full path, filename included, at 400 characters. “Decoded” just means OneDrive counts what you’d see in Explorer, not the URL-escaped version with %20 standing in for every space. Deeply nested folders plus long note titles get there faster than you would think.

The .obsidian folder with your settings and plugins is not on Microsoft’s blocked list, so it rides along. That means plugin settings sync between desktops, which is mostly what you want.

The duplicate file with your computer’s name in it

When OneDrive can’t reconcile two versions of a file, it keeps both and adds your computer’s name to one of them. That is where Daily Note-DESKTOP-7QK2M1B.md comes from. It does not ask. It is on you to notice, open both, and fold them back together.

Microsoft’s support page ties this symptom specifically to stale credentials and says to clear the saved ones. On Windows, search the Start menu for Credential Manager , open Windows Credentials , and remove the entries starting with MicrosoftAccount. On a Mac, open Keychain Access , search for OneDrive Cached Credential, and delete what comes up. If you keep getting computer-named duplicates on notes you only edited in one place, try that before blaming Obsidian.

The real cause is usually simpler: the same note edited on two machines before either finished syncing. Let OneDrive finish before you close the laptop lid. I said the same thing in the Dropbox sync post, because every folder-sync service fails exactly this way.

iPhone and iPad: Remotely Save or Nothing

Obsidian doesn’t officially support OneDrive vaults on iOS, and the vault-folder route that works on desktop is not an option there. So the answer on iOS is a plugin that syncs from inside Obsidian, and the plugin is Remotely Save.

The catch bites almost everybody, and the setup flow doesn’t warn you about it. On the free tier, Remotely Save’s README says it can only “read and write files in your OneDrive’s /Apps/remotely-save folder.” That’s the App Folder. If your vault already lives in Documents\Obsidian because Windows put it there, the free plugin can’t see it.

You have two options:

Move the vault into the App Folder. Close Obsidian first. Then, in File Explorer, move the vault folder into OneDrive\Apps\remotely-save\ (create it if it isn’t there yet). Right-click the vault in its new home and make sure Always keep on this device is still set, because a move is exactly the kind of thing that shakes a pin loose. Reopen Obsidian, use Open another vault → Open folder as vault , and point it at the new path. Then let Remotely Save on the phone sync against that. Free, and a little weird, because your notes now live in a folder named after a plugin.
Use Remotely Save PRO. PRO adds OneDrive with full access, which reaches the root of your drive, so the vault can stay where it is. PRO also adds smarter conflict handling that can merge small markdown changes instead of making a second file.

PRO needs its own account at remotelysave.com. It is “FREE while beta before Jan 1, 2027,” and the pricing page says “the price will be determined later.” I keep repeating that across this series because “free right now” and “free” are different words. If you pick PRO as your permanent answer, you are agreeing to a number nobody has announced.

The rest of the setup is the same as any other Remotely Save backend: install the plugin, pick OneDrive in its settings, authorize with your Microsoft account, and run the first sync with a backup of the vault sitting somewhere else. If you have never installed a community plugin, the plugin install walkthrough covers it, and the iPhone and iPad sync guide explains why iOS is the hard case for every method, not just this one.

Android: Remotely Save or a Sync App

Android is looser than iOS. Obsidian’s help page calls OneDrive’s Android support limited, but it doesn’t rule it out. You have two routes.

Remotely Save , exactly as above. Same App Folder limit on the free tier, same PRO option. If you already set it up on an iPhone, doing the same on Android is five minutes.

A folder sync app that mirrors a OneDrive folder to a real folder on the phone, which Obsidian then opens as a normal vault. The one built for this is OneSync: Autosync for OneDrive by MetaCtrl. The free version shows ads and caps uploads at 10MB per file, which a text vault will never hit and a vault full of PDFs will. Syncing more than one folder pair is part of the paid upgrade.

If you go the sync app route, where the vault lives on the phone matters. Obsidian on Android asks whether to keep the vault in app storage or device storage, and an outside sync app can only reach device storage. I covered that choice in the Android sync post.

Personal vs. Business: Your Work Account Is Out

If the OneDrive you have is the one your employer gave you, stop here for mobile.

The README’s wording: it “only works for ‘OneDrive for personal’, and not works for ‘OneDrive for Business’ (yet).” That’s true for both the free tier and PRO. The paid tier doesn’t add Business. It adds the root folder of a personal account.

Desktop is a different story. If the OneDrive client on your work laptop syncs a folder, Obsidian can open a vault inside it, and everything in the Files On-Demand section applies. Whether you should keep personal notes in your employer’s cloud is a question for you and your IT department, not for me.

The Cost Math If You Already Pay for Microsoft 365

This is the argument for OneDrive, and it holds up.

Microsoft 365 prices in the US, as of this writing:

Plan	Storage	Monthly	Yearly
Basic	100GB	$1.99	$19.99
Personal	1TB	$9.99	$99.99
Family	1TB each, up to 6 people	$12.99	$129.99

Personal and Family went up in January 2025 when Microsoft bundled Copilot into them, so if your mental number is older than that, it’s low.

Compare that to Obsidian Sync, which is $4 a month billed annually or $5 month to month for the Standard plan, with 1GB of storage and one vault. If you already pay for Microsoft 365 for Word and Excel, the storage is sunk cost and OneDrive sync is free. If you’re paying for Microsoft 365 only for the storage, Basic at $19.99 a year is still cheaper than Obsidian Sync. But you’re buying that cheaper storage with your own time.

OneDrive costs nothing extra on desktop. On phones it costs either a folder move, a PRO subscription with an unannounced price, or ads in a sync app. Obsidian Sync costs $48 a year and none of the above. So which one is cheaper? That depends on what an afternoon of your time is worth.

Should You Use It?

On Windows desktops, yes, if you already have it. Pin the vault folder, check Storage Sense, watch the filenames on your attachments, and it will mostly stay out of your way. Mac plus Windows works too, as long as you pin the vault on the Mac before you open it.

Once a phone is involved, it depends on the account. A personal OneDrive and an iPhone means Remotely Save, and the free tier means moving your vault into the App Folder. A work account and an iPhone means pick another method entirely.

I don’t sync my own vault with OneDrive. My laptops use Dropbox and my phone and iPad use Remotely Save, which I covered in the pillar guide. But if I were starting on a fresh Windows machine with a Microsoft 365 subscription, I would not go download something else just to avoid OneDrive. I would pin the folder and move on.

Whatever you pick, back up the vault somewhere that isn’t OneDrive. Sync is not backup. A tool that copies a deleted folder to every device you own in thirty seconds isn’t protecting anything.

Forced Connections: One Way to Get Original Ideas Out of AI

Stephan Miller — Wed, 30 Sep 2026 12:00:00 +0000

I have a real problem. I write a weekly roundup of which new AI models are worth paying attention to, and I want to turn it into an email newsletter. Which means I need people to subscribe to it, which means I need ideas for getting people to subscribe to it.

So I asked a model. Claude Sonnet 5, through OpenRouter, no system prompt, one line of context and then the question:

How can I get more readers to subscribe to it?

It came back with signup forms at the end of high-traffic posts, exit-intent popups, “publish 3-4 issues before heavily promoting,” a thread on X, r/LocalLLaMA, Hacker News, LinkedIn, newsletter swaps, and a single-field signup form. Every item was correct. I’ve read that exact list in maybe forty blog posts. It then asked me what my traffic source was so it could prioritize, which is the chatbot equivalent of the guy at the hardware store asking what you’re building.

Fine. That was a question, and a question gets you the average. So I asked it to be creative:

Give me creative, unusual, out-of-the-box ideas for getting more readers to subscribe to it.

This one was more fun. “Model Death Certificates,” obituaries for overhyped models. A “Which new AI model matches your vibe today?” quiz microsite. A referral leaderboard where the prize is absurd. A recurring mascot. An “anti-hype pledge.” It’s the list you get when you search “creative newsletter growth ideas.” I did not get creative ideas. I got the average of everything ever labeled creative.

Then I stopped asking it for ideas at all. I had a script pick an object at random from a list of twenty things. It picked three: a pressure cooker, a fire extinguisher and a metronome. I started with the fire extinguisher and told the model to list ten literal attributes of a fire extinguisher, force every single one onto my subscription problem, flag its own generic results, and keep what survived.

Two of the survivors:

Sell the silence. Market the newsletter as the thing you check only when a model actually matters. It came from “ignored until there’s a fire.”
Sell it as insurance. “Subscribe once. Skip every issue if you want. Just be covered when a model actually matters.” That one came from “reassurance from mere presence.”

Nobody writes that in a newsletter growth article, because it’s the exact opposite of what newsletter growth articles preach. It tells people they don’t have to read you. And I think it might be the best idea of the three runs.

The only thing that changed was that I stopped asking it a question and handed it a process instead.

What Just Happened
The Human Version (Run This First)
- 1. Write the problem as one line
- 2. Pick an object you did not choose
- 3. List ten literal attributes
- 4. Force every attribute onto the problem
- 5. Cross out everything you have read before
- 6. Do it with two or three objects, then breed the survivors
Handing the Model the Process
- What came out
- One caveat about the judge
Where This Came From
The Crossover Move
- Running it
Why Not Just Ask for Fifty Ideas?
Where It Breaks
So What Did I Actually Get?

What Just Happened

In the first post in this series I said a question gets you an answer and a move gets you something different. The obvious objection is that a “move” is just a more specific question. The second prompt above is the test case. “Give me creative, unusual, out-of-the-box ideas” is a more specific question. It got a more specific average.

The fire extinguisher did something different.

It did not supply the idea. A fire extinguisher knows nothing about newsletters. What it supplied was ten starting points that the model would never have started from, because none of them are anywhere near the words “grow a newsletter” in anything it was trained on. “Wall-mounted in a fixed location.” “Needs a periodic inspection tag.” “Different classes for different fires.” Each attribute forces the model to begin its reasoning somewhere other than the middle, and then go from there back to the problem. Some of the ideas end up back in the middle anyway. Some of them don’t.

You’re changing where the search starts.

The Human Version (Run This First)

This is a pen-and-paper technique and it has been one for close to seventy years. Do it once by hand before you ever hand it to a model, because the model version is just this process written down.

1. Write the problem as one line

Not a paragraph. “Get more people to subscribe to my newsletter.” “Name the new feature.” “Figure out what the second act of this story is.” If you can’t get it to one line you have two problems and should pick one.

2. Pick an object you did not choose

If you pick the object, you’ll pick one that already seems relevant, and relevance is exactly what you’re trying to escape. You can open a catalog to a random page, use the third thing to your left, a random word generator, the last noun on page 50 of whatever book is closest, or a list of twenty objects and a die.

Whatever it lands on, you keep it. No re-rolls.

3. List ten literal attributes

Literal. Not metaphors yet. Cover all of these:

Parts. What it’s made of, what you can see and touch.
Use. How you operate it, what it does, how long it takes.
Location. Where it lives, who’s near it.
Failure. What goes wrong with it, what wears out, what you have to maintain.
Feelings. How people feel about it. Nostalgia, fear, annoyance, indifference.

Do not skip the last two. I’ll show you why in a minute.

4. Force every attribute onto the problem

One idea per attribute, and take the attribute literally. “Needs a periodic inspection tag” becomes “send a literal quarterly ‘inspection’ email asking subscribers to reconfirm interest.” That’s the model’s actual output, by the way. Write it down.

The rule that makes the whole thing work: you are not allowed to skip an attribute because the mapping is awkward. The awkward ones are the point.

5. Cross out everything you have read before

Go down the list and mark every idea that could have appeared in a normal article about your problem. Most of them will. Whatever is left is what you came for.

6. Do it with two or three objects, then breed the survivors

One object gives you one set of starting points. Two or three give you survivors from different directions, and now: take two survivors from different objects and make one idea that needs both of them to exist. Not “pick the best one” or “improve one.” A child that neither parent could have produced alone.

Handing the Model the Process

Here’s the prompt I actually ran, word for word, with the object swapped in:

I write a tech blog and I'm starting a weekly email newsletter that rounds up
which new AI models are actually worth paying attention to. I want ideas for
getting more readers to subscribe. Don't answer that directly. Run this process
instead, and show every step:

1. The object is: fire extinguisher. List ten attributes of it. Physical parts,
   how it's used, where it lives, what goes wrong with it, how people feel about
   it. Literal attributes, not metaphors yet.
2. For each attribute, force a literal mapping onto the subscription problem.
   One idea per attribute. Do not skip an attribute because the mapping is
   awkward. The awkward ones are the point.
3. Mark any idea that could have come from a normal newsletter-growth article
   with [GENERIC]. Be honest.
4. Pick the two ideas that are least generic and still actually doable by one
   person, and say what the first concrete step would be for each.

Notice what’s in there and what isn’t. It never says “be creative.” It never says “think outside the box.” It never says “avoid generic ideas,” which is a whole separate problem I’ll get to later in the series. It’s steps 3 through 5 of the human version, written down in order, with the one line in the middle that stops the model from taking the easy exit.

I ran it three times with three objects: a pressure cooker, a fire extinguisher, and a metronome. Here’s a small script that does the same thing, if you want to run it against your own problem:

import json, os, urllib.request

MODEL = "anthropic/claude-sonnet-5"
PROBLEM = ("I write a tech blog and I'm starting a weekly email newsletter that "
           "rounds up which new AI models are actually worth paying attention to. "
           "I want ideas for getting more readers to subscribe.")

STEPS = """Don't answer that directly. Run this process instead, and show every step:

1. The object is: {obj}. List ten attributes of it. Physical parts, how it's used,
   where it lives, what goes wrong with it, how people feel about it. Literal
   attributes, not metaphors yet.
2. For each attribute, force a literal mapping onto the problem. One idea per
   attribute. Do not skip an attribute because the mapping is awkward.
   The awkward ones are the point.
3. Mark any idea that could have come from a normal article on this problem
   with [GENERIC]. Be honest.
4. Pick the two ideas that are least generic and still doable by one person,
   and say what the first concrete step would be for each."""

def ask(prompt):
    body = json.dumps({"model": MODEL, "temperature": 1.0,
                       "messages": [{"role": "user", "content": prompt}]}).encode()
    req = urllib.request.Request(
        "https://openrouter.ai/api/v1/chat/completions", body,
        {"Authorization": "Bearer " + os.environ["OPENROUTER_API_KEY"],
         "Content-Type": "application/json"})
    return json.load(urllib.request.urlopen(req, timeout=300))["choices"][0]["message"]["content"]

for obj in ["pressure cooker", "fire extinguisher", "metronome"]:
    out = ask(PROBLEM + " " + STEPS.format(obj=obj))
    open(obj.replace(" ", "_") + ".md", "w").write(out)

Swap in your own problem. Swap in your own objects, and don’t choose them yourself. That’s the part the script is for.

What came out

Thirty forced ideas. By the model’s own count, about ten of them were clearly not generic, three more were borderline, and the rest were, in the metronome run’s own words, “generic growth advice in a costume.”

The costumes tell you how the technique fails:

Attribute	Forced idea	Verdict
Pressure cooker: lid locks shut	A modal that seals the article until you subscribe or decline	Generic. It’s a content-lock popup.
Pressure cooker: builds pressure to cook faster	“Next batch replaces this list in 7 days”	Generic. It’s scarcity marketing.
Fire extinguisher: wall-mounted, fixed location	Subscribe box in the exact same spot on every post	Generic. Basic conversion advice.
Metronome: adjustable tempo weight	Pick a short or long version at signup	Generic. Everyone offers this.

Every one of those is a real attribute landing on something that already exists. The model went from a strange starting point and ended up back in the middle, the path of least resistance.

And the survivors:

Attribute	Forced idea
Fire extinguisher: ignored until there’s a fire	Sell the silence. Check it only when a model actually matters.
Fire extinguisher: reassurance from mere presence	Sell it as insurance. Skip every issue, you’re still covered.
Fire extinguisher: loud, messy discharge	One blunt, unhedged verdict per model, deliberately not balanced
Metronome: gets turned off out of annoyance	Ask everyone who unsubscribes one question and publish the answers monthly
Metronome: mixed feelings, eventually internalized and discarded	A graduation point. “Read 8 issues and you’ll be able to spot a hyped model yourself.”
Pressure cooker: emotional baggage, nostalgia or fear	A fixed narrator voice people get attached to, not just the information

Most of the survivors came out of the attributes about failure and feelings : ignored until there’s a fire, turned off out of annoyance, mixed feelings, emotional baggage. The physical attributes (the lid, the tempo weight, the wall mount) mostly mapped back onto tactics that already have names. It’s why step 3 in the human version tells you not to skip the last two categories. Parts are easy to list, and they’re also the ones that map onto what you already know.

The graduation point is my other favorite. Growth advice is built on keeping subscribers forever. That idea puts an end date on the relationship and uses the end date as the pitch.

One caveat about the judge

The model graded its own work, and a model grading its own originality is a pretty weak judge. It was also clearly being hard on itself on purpose and overcorrected. I’d rather it cut too much than let everything through. But the [GENERIC] flag is a filter, not a verdict. You still read the list yourself, and you still get the final say.

Where This Came From

The theory is Arthur Koestler’s. In The Act of Creation (1964) he argued that jokes, scientific discovery and art all run on the same machinery, which he called bisociation : two ways of seeing that each make perfect sense on their own and are never normally used together. Hold both at once and the collision is the idea. A pun is bisociation. So, in his telling, is a scientific breakthrough.

One of his best-known examples is Gutenberg. The part that’s documented is that the printing press adapted the screw press farmers already used for pressing grapes and olives. Gutenberg took a machine from the wine harvest and pointed it at metal type. That’s a clean case of two frames that came from two places. The tidier version, where Gutenberg has a flash of insight at a wine harvest and writes a letter about Minerva springing from his brain, is Koestler’s telling. Treat the combination as history and the eureka moment as a good story.

Charles S. Whiting’s Creative Thinking (1958) is where “forced relationships” is usually traced: take an item unrelated to the problem and force a connection anyway. An arbitrary thing, a forced mapping, and no permission to bail when the mapping gets weird.

It has a lot of relatives. Edward de Bono’s random-word technique is the same move with a word instead of an object. The “Combine” step in SCAMPER is a gentler version. Morphological analysis is its systematic cousin, where you break the problem into parameters and walk every combination instead of letting chance pick. I’ll get to that one later in the series. The distinction that matters here: forced relationships wants distance. The point is that the object has nothing to do with your problem.

The Crossover Move

So far this is one object at a time. The title of the post promises two things forced together.

Google DeepMind’s FunSearch (Romera-Paredes et al., Nature, December 2023) found new results in mathematics by having a language model write programs, scoring them, and keeping the good ones in a database. The interesting part is how it builds each new prompt. It doesn’t hand the model its best program and say “improve this.” It samples two programs from the database, sorts them by score, labels them v0 and v1, and asks for the next version. The paper found two programs worked better than one, with diminishing returns after that, and gives the reason in one sentence: combining several programs “enables the LLM to spot patterns across the different programs and generalize those.”

AlphaEvolve (Novikov et al., 2025) is the bigger, newer version of the same idea, with a program database built to keep the parents diverse so ideas explored earlier can resurface later instead of getting lost.

The evolutionary computation people have a name for this. It’s crossover. Two parents, one child. They’ve been doing it for decades and nobody in the prompt-template world seems to have noticed, because they call it “program search” instead of “brainstorming.”

FunSearch’s two parents come from the same island, so they’re related. It combines, but it doesn’t force distance. What I did below uses both halves: forced distance to generate the parents, crossover to merge them.

Running it

I took three survivors from two different objects and gave them back to the model:

Three ideas came out of separate exercises, each forced from a different
unrelated object:

Parent A (from a fire extinguisher): Sell subscribing as insurance. "Subscribe
once. Skip every issue if you want. Just be covered when a model actually matters."
Parent B (from a metronome): Build in an explicit graduation point. "Read 8
issues and you'll be able to spot a hyped model yourself."
Parent C (from a metronome): Publish churn. Ask everyone who unsubscribes one
question and publish the anonymized answers monthly.

Do not pick the best one and do not refine any one of them. Write three child
ideas. Each child must be a single mechanism that needs at least two of the
parents to exist, and could not be built from either parent alone. Name which
parents each child takes from and what it takes.

The “do not pick the best one and do not refine” line is doing the same job as “do not skip an awkward attribute” did earlier. Without it, a model handed three ideas will rank them and polish the winner, because that’s what a helpful assistant does with a list.

What came back:

Graduated coverage (insurance + graduation). At issue 8, a subscriber automatically drops to an insurance-only tier: silence unless a model genuinely matters. Graduation alone gives you no reason to stay after you graduate. Insurance alone has no moment where “skip whenever” becomes official.
The graduation exit log (graduation + churn). People who leave after issue 8 get a different exit question: not “why are you leaving” but “what made you confident enough to leave.” Those answers get published as proof the newsletter actually teaches something. Churn becomes a credential.
Churn to reinsurance (insurance + churn). The monthly published churn answers sit next to a one-click “reinstate your coverage” link. You were never really unsubscribed, just paused until it matters.

Graduated coverage is the insurance idea with a trigger attached, and the trigger is the thing the insurance idea was missing. I wouldn’t have gotten there from either object alone. And I definitely wouldn’t have gotten there from “give me creative ideas,” which got me a mascot.

Why Not Just Ask for Fifty Ideas?

This is the obvious shortcut, and it doesn’t work.

In 2024, Chenglei Si, Diyi Yang and Tatsunori Hashimoto ran a large study comparing research ideas from an LLM pipeline against ideas from over a hundred NLP researchers (arXiv:2409.04109). The LLM ideas were actually judged more novel than the humans’, and slightly less feasible. But to get there the pipeline generated 4,000 seed ideas per topic, and when they deduplicated them, only about 5% survived. The share of new, non-duplicate ideas in each batch kept dropping as they generated more, until it plateaued. The authors list the lack of diversity in generation as an open problem.

So asking for more doesn’t get you more. It gets you the same few ideas in different words, over and over, with the occasional new one. Volume is a terrible way to escape the middle.

Forcing a starting point is the cheaper way out. Thirty forced ideas gave me roughly ten survivors. Four thousand unforced ones gave that study about two hundred. The two aren’t directly comparable (different task, different judge, different everything), but the direction is the point.

Where It Breaks

I’m not going to pretend this is the only way to do it, because the research that exists doesn’t let me.

The one solid human study I found on distance points the other way. Joel Chan, Christian Schunn and colleagues looked at which sources of inspiration led to the most creative design ideas (Design Studies, 2015), and found that “conceptually closer rather than farther sources lead to more creative ideas,” consistently across different design problems. There was no support for the best ideas coming from the farthest sources.

That’s a different setup from this one. Their sources were examples of other solutions, and a fire extinguisher is not a solution to anything. It’s a jig, a thing to push your thinking against. In the runs above, the random object generated plenty of candidates and most of them were junk. The crossover step, merging survivors that were close enough to fit together, is where the best idea came from. That’s consistent with Chan’s finding, not a contradiction of it.

The other ways it breaks, from actually doing it:

Physical attributes map to existing tactics. A lid that locks becomes a popup. If your list is all parts, you’ll get all costumes. The failure and feeling attributes are where most of the survivors were.
The mapping can be too loose. If you let yourself (or the model) go metaphorical in step 2, anything maps to anything and the object stops doing work. Literal mappings are more constrained, which is what you want.
It solves a narrow kind of problem. It’s great when you’re stuck in a rut of five versions of the same idea. It’s useless when you don’t have enough information yet. No fire extinguisher is going to tell me whether anybody wants an AI model newsletter in the first place.
You still have to pick. Graduated coverage looks good to me because of what I know about the people who read this blog. The model doesn’t know any of that. The pick is yours.

So What Did I Actually Get?

A newsletter pitch I’d never have written on my own. It tells people they don’t have to read it, and it goes quiet after issue 8 for anyone who’s learned the skill. Does it work? I don’t know yet. The newsletter doesn’t exist yet.

The part that changed how I use models is smaller than that, though. I used to think the fix for a bland answer was a better question. It isn’t. What got me somewhere new was running a boring, seventy-year-old, pen-and-paper process, writing its steps down in order, and handing the model the steps instead of the question.

Next one: give yourself an arbitrary rule. It’s the most recommended creativity advice in existence, and there’s a reason to be suspicious of it.

Sonnet 5.5 vs Opus 5.5: The Cheap Claude Costs More

Stephan Miller — Tue, 29 Sep 2026 13:00:00 +0000

The price sheet caved this week. I don’t mean one lab and some polite matching. Anthropic and OpenAI both cut prices on September 22, a day after Xiaomi dropped a 28-cent open-weights model, and then Anthropic came back six days later with a new Sonnet it swears is cheaper to run. If you pay your own API bill, this was the best week of the year to be alive.

Then I read the fine print, because that’s the job. The new “cheap” Claude costs more per task than the new expensive Claude, at least when you turn the effort dial all the way up. The cheapest good coding model on the board this week comes from a company Anthropic says built it partly by siphoning Claude conversations through OpenClaw, which is the same agent framework I run in a Docker container in my house. And the free model sitting at number two on OpenRouter won’t tell you who made it.

So yes, prices went down. Read the meter anyway.

Everybody blinked at once
Opus 5.5 is the rare launch where every signal agrees
The cheap Claude costs more than the expensive Claude
OpenAI just showed up at the bottom of the price sheet
The cheapest coding model has a distillation problem
The cheapskate picks
The free model at number two won’t say who made it
Horror story: 48,218 files in 103 seconds
What’s coming
Finally

Everybody blinked at once

The clustering matters more than any one launch.

On September 21, Xiaomi shipped MiMo-V2.6-Pro and MiMo-V2.6-Flash as MIT-licensed open weights. Pro is a 1.02-trillion-parameter mixture-of-experts with 42 billion active, a million-token context, and it’s priced at 43.5 cents in and 87 cents out. Flash is 310 billion total, 15 billion active, and costs 14 cents in and 28 cents out. Artificial Analysis gives Pro a 46 on its Intelligence Index, which makes it the top-scoring open-weights model on the board right now.

A day later, on September 22, Anthropic released Claude Opus 5.5 at $4 in and $20 out, down from $5 and $25. Cache reads dropped 60 percent to 20 cents. Same day, OpenAI released GPT-6 Sol and GPT-6 Luna and cut prices in half or better. Sol went from $4/$20 to $2/$10. Luna went from 20 cents and $1.20 to 10 cents in and 50 cents out. OpenAI says those are permanent prices, not a promo, and credits caching and inference improvements.

Then on September 28, Anthropic released Claude Sonnet 5.5 at $2 in and $10 out. Same price as Sonnet 5, but a much better model.

Three labs from two countries, all pushing prices down inside 48 hours. I’ve been doing this roundup since April and I’ve never seen the whole price column in my spreadsheet go down at once like this. Usually one lab cuts and everyone else pretends not to notice for a month.

Opus 5.5 is the rare launch where every signal agrees

I spend a lot of this roundup explaining why the benchmark number and the blind-test number disagree. Three weeks ago GPT-6 Astra tied for number one on Artificial Analysis and debuted twenty-fourth on Arena. It’s twenty-sixth now. That split is normal. It’s practically the house style.

Opus 5.5 didn’t do that. It’s number one on the Artificial Analysis Intelligence Index at 58, five points clear of Fable 5.1 and GPT-6 Astra, which are tied at 53. It’s also number one on Arena Overall at 1509, and number one in Creative Writing, Instruction Following, and Hard Prompts. The hard-benchmark test and the blind vote landed on the same model in the same week. I almost never get to write that sentence.

It also ended a running gag. For weeks I’ve been pointing out that the outright leader in Instruction Following and Hard Prompts was claude-opus-4-6, a model about a year old, beating everything Anthropic shipped after it. Opus 5.5 finally took both categories. Coding is the one place the old guard hangs on: opus-4-6-high, opus-4-7-high, and fable-5-high are in a three-way tie at 1551, with Opus 5.5 at fifth.

Two asterisks. First, the votes are thin. Opus 5.5 has 2,307 votes in Overall and only 487 in Creative Writing, so its 20-point lead there could shrink as the crowd catches up. Second, every Claude score on Artificial Analysis still carries the “with fallback” tag, which means some answers came from a weaker model when the main one refused. I’ve flagged that on every Anthropic launch since Opus 5. The number is real. The footnote is load-bearing.

Reddit’s reaction has mostly been relief. The most-upvoted praise isn’t even about coding, it’s about how the thing talks. One r/ClaudeCode user called it “genuinely an order of magnitude improvement over Opus 5” in communication, which tracks. Opus 5 had a habit of saying “blast radius” and “load-bearing” like it was paid per use. (Yes, I just used “load-bearing” myself. I’m allowed. I’m not charging you per token.) Another user said they’d worked “non stop since release” and “barely made a dent” in their Max limits.

Not everybody agrees on that part. A HardForum user on the $100 Max plan said Opus 5 used to get them through whole days of coding, weekends included, without hitting the weekly limit. They upgraded to Opus 5.5 on Tuesday and were at 91 percent by Thursday morning. They were running it on “Extra” effort. Another user in the same thread, on medium, had used 12 percent of a $200 plan’s weekly limit, and a third summed it up: “5.5 extra eat tokens way more than previous gen.” Remember that effort dial. It comes back in a minute. Anthropic’s own demo was a HAProxy port from C to Rust that Opus 5.5 finished in 9.5 hours against Fable 5.1’s 12, at 51 percent lower cost. Both can be true. A well-defined port is not an open-ended feature in a messy real app, and the complaints are all coming from the messy real apps. And the best comment in the whole pile: “Queue the whining in two weeks about the model being nerfed.” Set a reminder.

The cheap Claude costs more than the expensive Claude

Sonnet 5.5 is the budget model. It’s half the per-token price of Opus 5.5. It’s also scary good. It scored 70.6 percent on Terminal-Bench 4.0, the command-line agent test, up from Sonnet 5’s 10.3, and ahead of Opus 5.5’s 66.4. On Artificial Analysis it scores 56, two points under Opus 5.5 at max and ahead of Fable 5.1 and GPT-6 Astra.

Then Artificial Analysis ran the whole index suite and checked the bill. At max effort, Sonnet 5.5 cost $7.60 per task. Opus 5.5 cost $5.98. The cheap model was 27 percent more expensive.

The reason is tokens. Sonnet 5.5 burned 410 million output tokens getting through the suite. The median for models in its price tier is 88 million. Opus 5.5 used about 260 million. At max effort, Sonnet thinks roughly 60 percent longer per task than Opus, and half the price per token doesn’t cover 60 percent more tokens. It’s a fast model, 138 tokens a second, and it spends that speed talking to itself.

It gets worse when you match scores instead of effort settings. Sonnet 5.5 at max and Opus 5.5 at xhigh both score 56 on the index. Sonnet costs $7.60 a task to get there. Opus costs $3.46. Same score, and the budget model costs 2.2 times as much. According to the same analysis, Sonnet only comes out cheaper at the bottom of the effort range.

None of that makes Sonnet 5.5 a bad model. It went from 10.3 to 70.6 on Terminal-Bench in one release, and one customer quoted by MarkTechPost measured about 121K tokens per answer against 497K on Sonnet 5. It’s just not automatically the cheap option anymore. So the practical advice is boring. Don’t crank Sonnet 5.5 to max because it’s “the cheap one.” If a task needs the top of the dial, Opus at xhigh gets you the same score for less. If it doesn’t, run either one at a lower effort and check your actual bill after a day, not the price page.

I’ve written some version of “cost per token is not cost per task” in this roundup maybe ten times. I’d never seen it flip the price order inside one lab’s own lineup before.

OpenAI just showed up at the bottom of the price sheet

For most of this year the cheapest-good-model story has been a Chinese open-weights story. Kimi, then MiMo, then GLM, and now maybe MiMo again. American labs competed at the top and left the floor alone.

GPT-6 Luna costs 10 cents in and 50 cents out. That’s exactly the list price of GLM-5.3-Flash, the model that’s been my cheapskate default for the last month. And Luna is in the Arena Coding band, ranked 43rd at 1515, only 36 points behind the leader. It isn’t the cheapest thing in the band, but it’s the first time I’ve seen an OpenAI model sit in a cheapskate band at the same price as the Chinese floor. It’s also fast. Artificial Analysis clocks it at 148 tokens a second, about three times GLM’s speed.

The catch is capability. Luna scores 37 on the Intelligence Index. GLM scores 42. Outside of coding, Luna falls out of the Arena bands entirely. It’s 86th Overall. So it’s a fast, cheap, preference-decent coding model, not a general replacement. But OpenAI ignored this end of the price sheet for a year, and now they’re in it.

GPT-6 Sol is the more awkward launch. $2/$10, Intelligence Index 48, and OpenAI’s own numbers have it beating Opus 5 on AutomationBench (33.2 percent vs 26.9) and edging it on OSWorld at 80 percent lower cost. On Arena it’s 60th Overall at 1457, which misses the Overall band by two points.

The cheapest coding model has a distillation problem

MiMo-V2.6-Flash is the cheapest model inside the Arena Coding band this week. It’s ranked 24th at 1525, 26 points behind the leader, for 28 cents out. That’s 89 times cheaper than a $25 leader. Artificial Analysis puts it on its intelligence-vs-cost Pareto frontier. On OpenRouter it went from nothing to the seventh most-used model on the platform in a week, 6.93 trillion tokens. By every signal I track, it’s a real value pick.

And on September 10, eleven days before it shipped, Anthropic published a threat intelligence report accusing seven China-based labs of what it calls “illicit distillation,” using Claude’s outputs to train their own models. Xiaomi was one of them. According to the-decoder’s summary of case GTG-16008, Xiaomi forwarded more than 400,000 conversations from users of its own MiMo chatbot to Claude between March and April, through OpenClaw and OpenCode, to pull out training data. Across all seven labs, Anthropic counts around 190 million exchanges. Xiaomi hasn’t responded.

I run OpenClaw. It’s the agent framework behind the assistant I run at home, the one that handles my scheduled jobs and writes into my notes vault. So I read that paragraph twice. To be clear, this is an accusation from a competitor, not a court finding, and nothing in it suggests OpenClaw itself did anything wrong. It’s a tool. Somebody pointed it at Claude at industrial scale. But if you were one of those 400,000 MiMo chatbot users, your conversations apparently took a trip you never agreed to.

So what do you do with the pick? I’m printing it, because the method is the method and the price and rating are real. I’m also printing it with two asterisks. First, it only has 1,043 Coding votes, so it’s the emerging pick, not the settled one. GLM-5.3-Flash is right behind it at 1523 with five times the votes for 50 cents, and that’s the steadier bet. Second, if where a model’s training data came from matters to your company, and for some of you it contractually does, this one has a question hanging over it.

And watch the price you see. On OpenRouter, MiMo-V2.6-Flash’s headline price shows 8 cents in and $1.28 out. That’s one small host’s endpoint. Xiaomi’s own endpoint is 14 cents and 28 cents, and it’s carrying 97 percent of the traffic. If you read the headline number, you’ll think it’s the most expensive cheap model on the board. Check the provider list.

The cheapskate picks

Same method every week. Take the category leader’s Arena rating, draw a band 50 points below it, and find the cheapest model still inside the band. The top of Arena is compressed, so the leader is usually only a little better than something 40 to 100 times cheaper. The band gets computed in code from the full table, never eyeballed off the first screen, because eyeballing is how you delete the whole cheap tail without noticing. I learned that one in public.

The good news: Arena finally refreshed. The last two issues ran on the same September 13 snapshot. This week’s data is stamped September 25. Bands ran deep: 57 models in Overall, 73 in Coding, 43 in Hard Prompts and Math, 38 in Instruction Following, and just 12 in Creative Writing. GLM-5.3-Flash is quoted at Z.ai’s list price, $0.15 in and $0.50 out. Arena’s price column now shows 20 cents out for it, but that’s a third-party floor, not list. Budget on fifty.

Category	Leader	$ out	Cheapskate pick	$ out	Δ rating	Cheaper by
Overall	claude-opus-5.5-high (1509)	$20	GLM-5.3-Flash (1474, #35)	$0.50	−35	40×
Coding	claude-opus-4-6-high (1551)	$25	MiMo-V2.6-Flash* (1525, #24)	$0.28	−26	89×
Creative Writing	claude-opus-5.5-high (1521)	$20	gemini-3.7-flash-high (1492, #4)	$3.75	−29	5.3×
Instruction Following	claude-opus-5.5-high (1516)	$20	GLM-5.3-Flash (1474, #26)	$0.50	−42	40×
Hard Prompts	claude-opus-5.5-high (1541)	$20	GLM-5.3-Flash (1499, #31)	$0.50	−42	40×
Math	claude-fable-5-high (1523)	$50	GLM-5.3-Flash (1501, #13)	$0.50	−22	100×

*Thin votes (1,043). The steadier Coding pick is GLM-5.3-Flash at #28, 1523, 50 cents, 50× cheaper.

GLM-5.3-Flash held four of six, and it’s doing it on real vote counts now: 19,103 in Overall and 12,503 in Hard Prompts. That’s a pick you can lean on. The “cheaper by” numbers shrank, from 100× to 40× in Overall and from 50× to 40× in Instruction Following and Hard Prompts, but not because GLM got pricier. The leader got cheaper. Opus 5.5 at $20 took the top of three boards from $25 and $50 models. That’s the price war showing up in my own table.

Coding is the first crack in GLM’s run in a month, and it’s a crack on thin votes. Math is still the row to squint at: GLM sits 13th on only 920 votes. If you want steadier, MiMo v2.5 Pro has 3,280 votes at 87 cents.

Creative Writing is where the method bit back. Gemini 3 Flash was the Creative pick for four straight issues at $3. It didn’t get worse this week. Opus 5.5 raised the ceiling by 17 points, and Gemini 3 Flash’s 1457 fell out of a band that now starts at 1471. When the leader gets better, cheap models fall out of the band without doing anything wrong. The new pick is Gemini 3.7 Flash at $3.75, ranked fourth on preliminary votes. Also, Gemini 3.8 Flash’s intro pricing doubles on January 1, so check whether 3.7 goes the same way before you build a budget around it.

The speed caveat is the same as every week, and the number moved again. Artificial Analysis clocks GLM-5.3-Flash at 48 tokens a second this week, down from 89 last week. MiMo-V2.6-Flash is 55. I’ve stopped carrying this number forward because it bounces around too much. Both are slow enough to notice in an agent loop. GPT-6 Luna at 148 is the fast one at this price.

The free model at number two won’t say who made it

Space Bunny Alpha showed up on OpenRouter on September 23 as a stealth model: no maker named, free, a million-token context, and text, image, and video input. Five days later it was the second-biggest model on the platform for the week at 18.2 trillion tokens, and number one for the most recent day.

It’s probably MiniMax. Tokenizer tests on launch day matched MiniMax’s M3 family on every string people threw at it (one tester ran 50 strings, all 50 matched), and on September 27 MiniMax released M3.1-Flash-Preview with the same context length, the same five reasoning effort levels, and the same fast-coding pitch. Nobody has officially confirmed it.

Two things worth knowing before you point production at it. It’s free, and free usage isn’t chosen usage. Free models in this roundup have a habit of spiking and then settling once the meter turns on. And the listing says prompts and completions “may be retained by the provider,” a provider that won’t tell you its name. The same week a distillation report dropped. Fine for kicking the tires on public code. I wouldn’t send it anything I’d mind reading in somebody’s training set.

Meanwhile, on the paid board, DeepSeek V4.1 Flash is number one at 20.8 trillion tokens, up 23 percent. GLM-5.3-Flash slipped to third at 13.3 trillion, down 21 percent, though it’s still number one for the trailing month at 57.5 trillion. Usage is moving toward whatever is newest, cheapest, or free, which is what usage always does in the week after a launch pile-up.

Horror story: 48,218 files in 103 seconds

Around September 20, a developer posted on r/ClaudeAI that Claude Code had deleted their project. According to TechRadar’s writeup, the agent was told to rebuild a mirror of the project for a task. It figured out build_mirror.py couldn’t refresh the mirror in place, so it wrote a little Python remover to delete an old copy sitting in a temp folder. That old copy held 7,332 ordinary files and 614 Windows directory junctions pointing back into the live project. The remover used os.walk with followlinks=False, which sounds safe. But os.path.islink() returns false for Windows junctions, so Python didn’t treat them as links, walked right through them, and started deleting the real project on the other side.

It took 103 seconds. It deleted 48,218 live files, and it emptied the Git object store too: .git/objects, refs, and logs. The index still listed 7,221 paths, but the actual file contents were gone, so Git couldn’t restore anything. TechRadar’s headline quotes the agent’s own summary: “I broke something.” (This all comes from the Reddit post and an attached report, not an independent forensic investigation, so treat the details as the poster’s account.)

Reddit’s verdict was harsh. It wasn’t wrong. The poster admitted they weren’t using GitHub properly and should have been working on a branch. But the reason I’m including it isn’t to dunk on someone. It’s that the agent did something that looks completely reasonable (clean up a stale temp copy) on a file system that had a trap in it that no human would have spotted in the moment either. Directory junctions and symlinks turn “delete this folder” into “delete whatever this folder points at,” and on Windows the standard Python safety check doesn’t even see the junctions. If you let an agent run rm or its Python equivalent, the only real protection is a remote it can’t touch, so push before you let it clean up anything.

What’s coming

Claude Haiku 5.5 is next. Anthropic said Sonnet and Haiku 5.5 would follow Opus “in the coming weeks.” Sonnet already shipped. Haiku is the one left, and if it’s priced like Haiku usually is, it lands right in the cheapskate zone.

MiniMax M3.1 should get official soon. If Space Bunny gets unmasked and priced, we’ll see how many of those 18 trillion free tokens stick around once there’s a bill.

Then there’s MiMo-V2.6-Pro-UltraSpeed, which Xiaomi says is up to 20 times faster than Pro at the same quality. It already has 73 billion tokens on OpenRouter. If that holds, “cheap but slow” stops being the standard caveat on the Chinese value picks.

DeepSeek V5 is speculation only. People keep floating October. DeepSeek hasn’t said anything, and I’m not putting a date on a model that doesn’t have one.

Finally

The price war is real, and it’s good for you. Opus got 20 percent cheaper and better at the same time. OpenAI cut Sol and Luna in half and made it permanent. The floor got a new 28-cent option. If you’re paying your own bill, almost every line on your invoice should drop next month. Who had “two frontier labs cut prices on the same day” on their bingo card?

But three things I’ve been harping on all year showed up again, just in cheaper clothes. The sticker isn’t the task: the budget Claude costs more than the premium Claude when you run it hot. The free model isn’t free: someone is paying for those 18 trillion tokens, and they’re not telling you who or why. And the cheapest option comes with questions about where it came from, which some of you will care about and some of you won’t.

Last week I said I’d come back to see whether the Arena crowd was any nicer to Grok 4.7 than its creator was. It wasn’t. Grok 4.7 debuted at 92nd Overall. Its creator called it mid. The crowd called it worse. Low bar, and it still found a way under it.

Track New AI Models on the Arena Leaderboard With a 70-Line Scraper

Stephan Miller — Thu, 24 Sep 2026 12:00:00 +0000

Every week I write the Model Buzz Report, and every week part of the job is staring at the Arena leaderboard trying to remember what it looked like last week. Which of these is new? Was that one there before? When did GLM 5.3 show up in the top 20, because it sure felt like it came out of nowhere?

That is a question a scraper cannot answer. A scraper hands you the page as it is right now, and “what’s new” is a comparison between now and then. You need a then.

So this post is about the then. It is the smallest useful version of the thing most scraping recipes in this series is going to need: save each run with a timestamp, run it on a schedule, and compare this run with the last one. The example is deliberately easy, one page and one question.

Why “Now” Is Almost Never the Question
The Page: Arena’s Text Leaderboard
The Harness, All 70 Lines
- Storage: SQLite, Not a JSON File
- The Diff: Two Kinds of “New”
- The Schedule: One Cron Line
Day One Has No Then
Two Things It Got Wrong
- That “NEW #1” Is a Rename
- It Missed the One That Started This
Why Not Just Use Firecrawl’s Change Tracking?
What This Is For

Why “Now” Is Almost Never the Question

Think about what people actually want out of a scraped page. Is this price a deal? Did the competitor change their pricing? Which models are new near the top? None of those can be answered from one snapshot. “Is this a deal” needs the price history. “Did they change” needs the old page. “What’s new” needs the old list.

Firecrawl solves the ugly part of scraping, which is getting a clean page out of a site that renders everything with JavaScript and would rather you went away. It does not solve the then. Nothing that fetches a page can, because the then is data you had to collect yourself, back when it was the now.

That takes three boring things:

Storage. Save each crawl with a timestamp instead of printing it and forgetting it.
A schedule. One crawl tells you nothing. A cron line fixes that.
A diff. Compare this crawl with the last one and only say something when something changed.

That is the whole harness. I keep wanting to call it a framework and it keeps being 70 lines.

The Page: Arena’s Text Leaderboard

The target is arena.ai/leaderboard/text, the overall text leaderboard. About 400 models, ranked by head-to-head human votes, with score, vote count, price, and context window per row.

It is also a page that doesn’t want to be read by a simple fetch. The table is rendered client-side, and when I built the Model Buzz skill I learned that some fetch tools hand back the wrong leaderboard on category URLs without any error at all. That is the worst kind of failure: data that looks plausible.

Firecrawl got it clean on the first try:

firecrawl scrape https://arena.ai/leaderboard/text --only-main-content --wait-for 3000

What comes back is markdown, and the leaderboard is a plain markdown table:

| Rank | Rank Spread | Model | Score | Votes | Price $/M | Context |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | 17 | Anthropic<br>[claude-fable-5-high](https://www.anthropic.com/news/claude-fable-5-mythos-5)<br>Anthropic · Proprietary | 1506±5 | 30,057 | $10 / $50 | 1M |
...
| 19 | 737 | [glm-5.3-max](https://z.ai/blog/glm-5.3)<br>Z.ai · MIT | 1483±6 | 10,960 | $1.40 / $4.40 | 1M |

I gave it three seconds to let the page render. I didn’t test whether it needs that, so treat the number as superstition you are free to delete.

No JSON schema, no LLM extraction, and no selectors. A regex reads that table fine. A plain scrape is one credit a page, and Firecrawl’s JSON extraction adds four more on top, so asking an LLM to parse a table this regular would be paying five times over for something a regex already does. If you haven’t installed the CLI yet, the setup post covers it and the installer you should skip.

The Harness, All 70 Lines

Python, standard library only, calling the Firecrawl CLI through subprocess. No SDK, so the only install is the CLI you already have.

#!/usr/bin/env python3
"""Watch the Arena text leaderboard and report models that just showed up near the top.

    python3 arena_watch.py # scrape now, save, compare with last run
    python3 arena_watch.py --url <wayback url> --at 2026-09-02 # backfill an old snapshot
"""
import argparse, re, sqlite3, subprocess, sys
from datetime import datetime, timezone

URL = "https://arena.ai/leaderboard/text"
DB = "arena.db"
TOP = 20

ROW = re.compile(r"^\| (\d+) \| \d+ \| (.+?) \| (\d+)±")
NAME = re.compile(r"\[([^\]]+)\]")

def scrape(url):
    out = subprocess.run(
        ["firecrawl", "scrape", url, "--only-main-content", "--wait-for", "3000"],
        capture_output=True, text=True, check=True).stdout
    rows = []
    for line in out.splitlines():
        m = ROW.match(line)
        name = m and NAME.search(m.group(2))
        if not name:
            continue # skips the calendar widget and anything else that isn't a model row
        org = m.group(2).split("<br>")[-1].split(" · ")[0]
        rows.append((int(m.group(1)), name.group(1), org, int(m.group(3))))
    return rows

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--url", default=URL)
    ap.add_argument("--at", help="timestamp to record, for backfilling old snapshots")
    args = ap.parse_args()

    rows = scrape(args.url)
    if len(rows) < TOP:
        sys.exit(f"only parsed {len(rows)} rows, the page layout probably changed")

    taken = args.at or datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M")
    db = sqlite3.connect(DB)
    db.execute("""CREATE TABLE IF NOT EXISTS ranks
                  (taken TEXT, rank INT, model TEXT, org TEXT, score INT)""")
    db.executemany("INSERT INTO ranks VALUES (?,?,?,?,?)", [(taken, *r) for r in rows])
    db.commit()

    prev = db.execute("SELECT MAX(taken) FROM ranks WHERE taken < ?", (taken,)).fetchone()[0]
    if not prev:
        print(f"{taken}: first run, saved {len(rows)} models. Nothing to compare yet.")
        return

<img src="/images/2026/track-new-ai-models-on-the-arena-leaderboard-body-2.jpg" alt="The Harness, All 70 Lines" srcset=" /assets/resized/480/track-new-ai-models-on-the-arena-leaderboard-body-2.jpg 480w, /assets/resized/800/track-new-ai-models-on-the-arena-leaderboard-body-2.jpg 800w, /assets/resized/1400/track-new-ai-models-on-the-arena-leaderboard-body-2.jpg 1400w, " loading="lazy">

    was_top = {m for (m,) in db.execute(
        "SELECT model FROM ranks WHERE taken = ? AND rank <= ?", (prev, TOP))}
    seen = {m for (m,) in db.execute("SELECT DISTINCT model FROM ranks WHERE taken < ?", (taken,))}

    print(f"{taken} vs {prev}")
    for rank, model, org, score in rows:
        if rank > TOP:
            break
        if model not in seen:
            print(f" NEW #{rank:<3} {model} ({org}, {score})")
        elif model not in was_top:
            print(f" MOVED UP #{rank:<3} {model} ({org}, {score})")

if __name__ == " __main__":
    main()

Here is what each piece is doing, mapped to the three boring things.

Storage: SQLite, Not a JSON File

Every run inserts every row, all 400 or so, stamped with the time it was taken. One table, five columns.

I went back and forth on JSONL here. A line of JSON per run is simpler to look at and fine for a diff against the last run. But the questions I actually care about later are history questions: how long has this model been in the top 20, when did it first appear anywhere on the board, is it climbing or sliding. With JSONL you write a loop for each of those. With SQLite you write a query. And SQLite ships with Python, so it costs nothing to install.

Saving the whole board instead of just the top 20 is on purpose. Storage is cheap and you can’t go back and scrape last Tuesday. A model that debuts at #24 is not news today, but the day it cracks the top 20 you want to know it has been lurking for a week.

The Diff: Two Kinds of “New”

The script reports two different things, and the difference matters.

NEW means the model has never appeared anywhere on the board in any previous run. This is the one I actually wanted. A brand new model landing in the top 20 on its first appearance is the “where did that come from” moment.
MOVED UP means the model was on the board before but was not in the top 20 last run. A climber. Less exciting, still worth a look.

Everything else, a model shuffling from #7 to #9, stays quiet. The whole point of the diff is that most runs should print almost nothing.

The Schedule: One Cron Line

0 8 * * * cd /path/to/arena-watch && python3 arena_watch.py >> watch.log 2>&1

Daily at 8am, appended to a log. That lives on my mini PC, which is on all the time. On a Mac you can use cron too, or launchd if you enjoy writing XML. Once a day is plenty for a leaderboard that needs thousands of votes to move a model. Each run is one scrape, which on Firecrawl is one credit.

Day One Has No Then

Here is the catch with every history-based tool: the first run is useless. It saves 400 rows and prints “Nothing to compare yet.” You have to wait a day for the second run before the thing does anything at all, and a week before it does anything interesting.

I didn’t want to wait a week to write this post, so I cheated with the Wayback Machine. The Internet Archive snapshots the Arena leaderboard most days, and Firecrawl will scrape an archived copy just like the live page. That is what the --url and --at flags are for: point the script at an old snapshot and tell it what date to record.

python3 arena_watch.py --url "https://web.archive.org/web/20260902190730/https://arena.ai/leaderboard/text" --at "2026-09-02 19:07"
python3 arena_watch.py --url "https://web.archive.org/web/20260909040114/https://arena.ai/leaderboard/text" --at "2026-09-09 04:01"
python3 arena_watch.py --url "https://web.archive.org/web/20260917153328/https://arena.ai/leaderboard/text" --at "2026-09-17 15:33"
python3 arena_watch.py

The archived pages parse the same as the live one, since the table structure is identical and only the link URLs change. Three weeks of history in four commands:

2026-09-02 19:07: first run, saved 399 models. Nothing to compare yet.
2026-09-09 04:01 vs 2026-09-02 19:07
  NEW #3 claude-fable-5.1-max (Anthropic, 1504)
2026-09-17 15:33 vs 2026-09-09 04:01
  NEW #8 muse-spark-1.3-max (Meta, 1493)
2026-09-23 01:27 vs 2026-09-17 15:33
  NEW #1 claude-fable-5-high (Anthropic, 1506)

Fable 5.1 Max debuting at #3 and Meta’s Muse Spark 1.3 Max at #8 are exactly the kind of thing I wanted flagged. Midway through doing this the Internet Archive went down with a “Temporarily Offline” page, which is a nice reminder that the backfill trick is a trick and not infrastructure.

Two Things It Got Wrong

That “NEW #1” Is a Rename

Look at the last line again. claude-fable-5-high at #1 as a brand new model. It is not. The database says so:

sqlite3 arena.db "SELECT model, taken, rank, score FROM ranks WHERE model LIKE 'claude-fable-5%' ORDER BY model, taken;"


claude-fable-5|2026-09-02 19:07|1|1508
claude-fable-5|2026-09-09 04:01|1|1507
claude-fable-5|2026-09-17 15:33|1|1506
claude-fable-5-high|2026-09-23 01:27|1|1506

Same rank, same score, new name. This is the most common way change detection lies to you, and it is not a Firecrawl problem or a SQLite problem. It is an identity problem: the thing you are tracking needs a stable key, and the page does not give you one.

The cheap fix is a sanity check before calling anything NEW: if an unseen name sits at the exact rank and score of a name that just disappeared, call it a rename. I left it out of the script because 70 lines that lie once a month teach more than 90 lines that hide it. Your mileage may vary once it wakes you up at 8am about a model that is three months old.

It Missed the One That Started This

GLM 5.3 Max, the model that made me want this in the first place, never shows up as NEW. It was already sitting at #18 on September 2, the oldest snapshot I loaded, so as far as the database knows it has always been there:

2026-09-02 19:07|18|1482
2026-09-09 04:01|20|1482
2026-09-17 15:33|19|1483
2026-09-23 01:27|19|1483

The harness can only see changes that happen after it starts watching. You can’t get the then after the fact. Start the cron before you need it.

Also notice GLM wobbling from #18 to #20 to #19 while its score barely moves. Arena ranks a lot of models within a few points of each other near the top, and a model right on the #20 line will flicker in and out. Had it dropped to #21 for one run, the next run would have called it MOVED UP. If that gets noisy, the fix is to compare against the top 20 from any of the last few runs instead of just the last one.

Why Not Just Use Firecrawl’s Change Tracking?

Fair question, since Firecrawl has a changeTracking format and a whole Monitor product built around scheduled checks. I didn’t use either here, and the reason is the leaderboard itself: the vote counts on every row change constantly, so “did this page change” is always yes. What I want to know is not whether the page changed but what changed in one column of one table, compared against a history I can query.

That is not a knock on those features. For a page that should be static, a pricing page or a terms of service, “tell me when this changes” is exactly the right tool. They are worth their own post. This is the case where you want your own history, because the question you ask of it next month is not one you know yet.

What This Is For

As a standalone tool this is a small convenience. I will run it next to the Model Buzz Report and it will save me the “wait, was that there last week” squint.

The reason it is the first real recipe in this series is the shape. Scrape, save with a timestamp, compare with the last run, speak only when something changed. Every recipe coming after this one is that shape pointed at a different page with a different question: price history instead of rank history, a job board instead of a leaderboard. Firecrawl handles getting the page. These 70 lines handle remembering it.

And the database is already there, filling up once a day, waiting for whatever question I think of next.