DEV Community: Aleksei Grebenkin

I Didn't Believe a Chart, So I Rebuilt It From the Source Data

Aleksei Grebenkin — Thu, 03 Sep 2026 17:33:55 +0000

I saw a chart on Twitter ranking cities by cost of living against quality of life. New York and Los Angeles were sitting near the bottom of the quality axis, below places I could barely put on a map.

That is not what you expect to see. I did not believe it, and the fastest way to settle that kind of argument is to pull the source tables and rebuild the thing yourself.

It took a lot longer than I thought. Not the chart — the chart is the easy part. Collecting the data and then checking it is where the time went, and the interesting failures are all in there.

Three sources, one circle

The rebuild uses three open sources for 303 cities.

Numbeo, mid-year 2026 release — nine metrics: safety, health care, commute time, pollution, climate, purchasing power, property price to income, and two cost-of-living indices. I deliberately took a fixed half-year snapshot instead of the live "current" numbers, so a reader coming back next month sees the same chart I described.

OpenStreetMap, August 2026 snapshot — everything Numbeo does not measure, counted inside a circle of 10 km around each city centre: museums, galleries, theatres, cinemas, bars, pubs, clubs, rail stops and tram stops, pedestrian streets, playgrounds, parks, schools, libraries, universities. The circle is identical everywhere, 314 km², which means a raw count is already a density and I never need population data.

OurAirports plus Wikipedia — the number of unique nonstop destinations from every airport within 60 km, merged and counted once.

Everything is scripted, which does not make it fast: thousands of requests, then a reconciliation pass, then the part nobody budgets for — checking that the numbers are actually about the city you think they are.

What broke

The geocoder returned an administrative centroid. For Cebu and Gdansk it handed back the centre of the administrative unit rather than the city, so the 10 km circle landed in a field somewhere and every OSM metric for those cities collapsed to almost zero. This one caught itself: a city can lose one metric plausibly, but it cannot lose all of them at once. Coordinates for those two are now pinned by hand.

Diacritics failed silently. Wikipedia page titles with accented characters broke the request without raising anything, so Prague, Krakow, Atlanta and Minneapolis each came back with zero nonstop destinations. A zero looks like data. That is what makes it dangerous — a crash you notice, a plausible number you do not.

A dataset field contained something entirely different. In OurAirports, the wikipedia_link field for Dhaka's Hazrat Shahjalal International Airport holds a link to somebody's LinkedIn profile. The parser followed it, found no article, and moved on with nothing. The fix is a fallback that searches for the airport by name — but the lesson is broader: in an open dataset that anyone can edit, a field can contain anything at all, and your code should treat "wrong type of content entirely" as a normal case rather than an impossible one.

The cache poisoned the retry. Empty results were written to the cache before the fallback path ran, so the fallback then read the empty result back and confirmed it. Caracas got permanently stuck on a disambiguation page this way. Cache successes, not failures.

Same-name cities shared a key. London, Ontario overwrote London, UK. San Jose, California overwrote San Jose, Costa Rica. The key is now city plus country, and both twins carry the country in their label.

Recounting flights afterwards changed the typical city by about 4%, but it fixed the ones that were badly wrong: Charlotte went from 15 destinations to 200, Dallas from 94 to 271.

The formula was harder than the code

The first version of the score dropped purchasing power and price-to-income entirely. Numbeo's own quality-of-life index includes both, and if you leave money on the vertical axis while cost of living is on the horizontal one, the chart mostly measures itself: rich cities score high by construction.

I have since put both back — as options, not as my decision. They do measure something the cost index does not: what a local salary actually buys, and how many years of income a flat costs. That is a different question from what a basket of groceries costs, and it belongs to whoever is reading, not to me. It is off by default and there is a warning attached to it, because it is also the softest number in the set.

Three things people asked for and did not get. Bureaucracy — I have no idea how to measure it honestly, and a number I cannot defend is worse than a missing column. City size — that one is a fair request and it is on the list. Trends over time — this is a genuinely large collection problem: the historical series are scattered, they need reconciling and normalising across sources, and I do not think the result would pay for the work. Feasible for one city, miserable for 303.

What I left broken on purpose

Every weak spot is written on the page instead of quietly smoothed over.

Numbeo's purchasing power is self-reported by whoever fills in the form, and the sample skews to expats and urban professionals. Hyderabad comes out second in the world, above Zurich, while Mumbai sits at 63 — a fourfold split inside one country is a sampling artefact, not an economy.

In France a "college" is a school for eleven-year-olds, so Paris reports 407 of them against 149 universities and French cities look over-educated. Playgrounds are mapped far more densely in Europe than in Asia, and I could not separate real difference from mapping habits. Kindergartens were dropped entirely for the same reason: the tag is barely used in the US, and Europe came out thirteen times more child-friendly than North America.

There is no correct weighting

That is the actual conclusion, and it took a few hundred comments to make me build the obvious thing.

The chart exports itself with your settings written underneath, so any version of it can be shared as an image.

Any index like this is a weighted sum, and the weights are a value judgement wearing a lab coat. One person wants clean air, another wants a bar within walking distance, a third only cares that it is warm in February. So the twelve metrics now have twelve sliders, and the vertical axis measures whatever you turned up. The whole configuration lives in the query string, which means the chart you built is a link you can send to someone.

My own setup weights culture and car-free living highest, then affordability and safety. The city that comes out on top is Tokyo — expensive, and still worth it on my weights. I did not expect that, and I would not have found it with anyone else's formula.

The chart is here: https://avgrebenkin.com/research/cities/

The Work Didn't Disappear. It Moved Into Review.

Aleksei Grebenkin — Fri, 14 Aug 2026 12:32:48 +0000

Everyone measuring AI-assisted development is reporting the same shape of result: output is up, and the cost of checking that output is up much more. I want to walk through the numbers, because the gap between the two is larger than most teams have adjusted for, and then say what it looks like from inside a small team where I do most of the reviewing myself.

What the telemetry says

Faros AI published an engineering report built on two years of telemetry from 22,000 developers across more than 4,000 teams. Not a survey about how AI feels, but workflow data before and after AI adoption inside the same organisations.

The delivery side improved:

task throughput per developer up 33.7%
epics completed up 66.2%
pull request merge rate up 16.2%

The review side moved differently:

median time a pull request sits in review up 441.5%
average review time up 199.6%
median time to first review up 156.6%
daily pull request contexts per developer up 67.4%

Two numbers in the same report describe what happens when that pressure has nowhere to go:

pull requests merged without any review up 31.3%
incidents-to-PR ratio up 242.7%

So teams write more, and the queue in front of the reviewer grows faster than the queue behind the author. Some of that queue gets resolved by not reviewing at all, and the incident numbers follow.

What developers say about it

Sonar's State of Code Developer Survey, published in January 2026 with more than 1,100 developers, adds the human half:

96% do not fully trust that AI-generated code is functionally correct
only 48% always check AI-assisted code before committing
38% say reviewing AI-generated code takes more effort than reviewing code written by a colleague
AI accounts for 42% of committed code

Read the first two together and you get the shape of the problem. Almost nobody trusts the output, and barely half of them verify it. The trust gap does not turn into review work automatically. It turns into review work for the people who have the discipline or the responsibility, and into risk for everybody else.

That 38% is the number I recognise most.

Why checking is more expensive than writing

When you write code yourself, the mental model builds as you go. By the time the last line is in place you already hold the reasons behind every decision, including the ones you rejected. Review is not that. Review is reconstructing a model backwards from a finished artefact, without the discarded branches, without the moment where the author decided this approach was worth it.

That is expensive with a colleague's code, and it is more expensive with a model's code, because a model produces plausible work uniformly. A colleague signals uncertainty. Their commit message hedges, they leave a comment, they ping you about the part they are unsure of. Generated code arrives with the same confident surface everywhere, so the reviewer has no gradient to follow and has to give every section the same level of attention.

Add the context switching. A 67.4% increase in daily pull request contexts means the interruptions do not just multiply, they arrive on someone else's schedule. Work that runs on its own still comes back to you, and it comes back at a moment you did not choose. Each return costs the reload of a context you had already put down.

What this looks like on a small team

I build a product mostly by myself, with agents doing a growing share of the mechanical work. Automation ate the typing. It did not touch the checking, and it multiplied the number of things waiting to be checked.

I still cannot take a result from a model and send it to production without looking at it. Not once has that felt safe. So a layer of review always stays, and that layer is where the day actually goes. I have used AI coding tools for a long time, but the load only became obvious once individual completions turned into a set of automations running in parallel. The bottleneck stopped being how fast I write and became how fast I can rebuild context for work I did not do.

I do not have a clean set of rules that solves this. The things that visibly help me are unglamorous:

smaller units of generated work, because review cost grows faster than diff size
fewer parallel threads, even at the cost of throughput, because each extra thread taxes every other one on return
tests written before generation, so a chunk of verification runs without me holding the whole model in my head
treating "I do not understand why this works" as a blocking result, not a stylistic complaint

None of that closes the gap. It moves it a little.

The part worth arguing about

The industry conversation is still mostly about generation quality, on the assumption that better models shrink the review burden. The telemetry suggests review load is not a bug in current model quality but a structural property of delegating work you remain accountable for. Even at high accuracy, someone has to hold responsibility, and holding responsibility means rebuilding context.

That is why the productivity number and the review number in the Faros report should be read as one result and not two. Throughput up a third, review time up more than four times over. If your planning captured the first number and not the second, your team is absorbing the difference somewhere: in senior engineers' evenings, in merges nobody looked at, or in the incident count.

Where does your review time actually go, and what have you changed to make it survivable?

Sources: Faros AI, "The AI Engineering Report 2026: The Acceleration Whiplash" (telemetry from 22,000 developers across 4,000+ teams). Sonar, "State of Code Developer Survey 2026" (1,100+ developers, published January 8, 2026).

AI Watermarks Can't Prove Who Wrote Anything — and That's the Point Everyone Misses

Aleksei Grebenkin — Thu, 13 Aug 2026 06:34:22 +0000

Within 48 hours at the turn of August, both major AI labs switched on watermarking. On July 31, OpenAI embedded SynthID marks into all GPT-Live voice output — one day before Article 50 of the EU AI Act came into force. From August 2, Anthropic began weaving an imperceptible watermark into everything Claude writes.

My first thought was that this is very hard to do in a way that actually holds. Text has no pixel layer to hide anything in. So I read how Anthropic plans to do it — and dug into how text watermarks work in general. The honest answer: it works, and it also breaks in a lot of ordinary situations.

How do you even hide a mark in plain text?

There are three generations of this idea. (A good breakdown of the mechanics went viral this week in a thread by ML engineer Daria Berezhnaia — I'm borrowing her framing for the first two.)

Zero-width Unicode. The ancient way: invisible characters tucked between words. Paste the text into Notepad and copy it back out — the mark is gone. Nobody serious uses this anymore.

Token-level (statistical) watermarking. When the model generates text, many word choices are near coin-flips: "the cat sits" vs "the cat lies" might both be ~50% likely. A secret key decides which of the equally-probable options the model picks, again and again across the text. Where the model is confident — 90% one option — the choice is left alone, so quality barely suffers. A detector with the key checks whether word choices are "skewed" in the right pattern. This is the academic classic (green/red token lists), and Anthropic's "embedded in the text itself" wording points this way.

Embedding-space watermarking. The subtler one. Inside the model, every step produces hidden states — vectors that trace how the model "thought". You can nudge those vectors slightly in a secret direction; word choices shift almost imperceptibly, nothing visible at the text level, but the statistical fingerprint is there. Research is now pushing this toward semantic clusters, precisely so that paraphrasing won't wash the mark out.

Where it stops working

Taken mostly from Anthropic's own documentation, plus what follows from the mechanics:

Short text. Their wording: a very short passage leaves "too little text for a reliable signal". A mark spread across word choices needs volume, so a headline, a chat reply or a commit message is out of reach.
Editing. Heavy editing, paraphrasing, translating, or mixing the output into your own writing all thin the signal out until it is gone.
Older models, until the transition period for pre-August releases finishes.
Screenshots and format conversion, which strip file metadata completely.
Only the vendor can check. The key is secret, so there is no independent verification: Anthropic tells you whether Claude touched a text; you cannot audit the claim, and false positives on short "skewed-looking" text are a real statistical possibility. The detection tooling and its documentation haven't even shipped yet.

The part I did not expect

A detected mark says Claude processed the text, not that Claude wrote it. Anthropic states this plainly. Proofreading, translating and summarising leave the same mark as generating from scratch. I dictate my posts and run them through a model to fix grammar, so the thinking is mine and the mark would be there anyway.

Read it the other way and it is no stronger. No mark found proves nothing.

So the people this catches reliably are the ones who paste raw output and change nothing. Anyone who edits carefully, translates, or runs the text through a second model comes out clean — today there is no second embedding-space mark to replace the first one, and even when every lab ships one, each key stays locked in its own vault.

The voice side has the same hole — plus one more

OpenAI's audio watermarking repeats the pattern: their Verify tool answers one question — whether audio was "made by OpenAI" — not who authored the words. And voice adds its own escape hatch: the moment marked audio is transcribed to text, the audio watermark is gone. Speech-to-text is the analog hole of voice provenance.

So is it pointless?

I do not think so. A weak provenance signal beats no signal, someone had to move first, and Anthropic published the limitations instead of burying them. It just cannot carry the weight people will want to put on it, which is proof of who wrote something.

Would you want your own writing marked because you asked a model to fix your commas?

Sources: Anthropic — How Claude marks AI-generated content · thread on watermark mechanics by @dariadsss · OpenAI adds SynthID to GPT-Live voice · token-level watermarking: Kirchenbauer et al., "A Watermark for Large Language Models" · embedding-space: PASA, SimMark

About: I'm Aleksei, a software engineer. I dictate most of what I write.