DEV Community: jack

AI cost modelling: measure your own token counts, the published table was off by 2.3x

jack — Thu, 24 Sep 2026 16:32:50 +0000

We resell image generation, so we have to know what a generation costs us before we can price it. That turned out to be a lot harder than reading the pricing page, and the two times it bit us were both cases where a number looked authoritative and was not.

Ratios taken from the published table were off by 2.3x on one tier

The provider charges per token. Tokens per image depend on resolution and on a quality setting, and the pricing page does not list tokens per image. It points you at a calculator in the docs, built from a table of token counts for a 1024×1024 image.

So we took our quality multipliers from that table. Low costs about 0.26 of Medium, High costs about 3.94 of Medium, price accordingly, move on.

To be fair to the provider, that table is not wrong. It describes one configuration, correctly. The mistake was ours: we treated ratios from a single 1024×1024 case as if they were the model's quality multipliers everywhere.

Then we measured. Nine generations, three resolutions, three quality settings, reading the token counts back out of our own usage records:

	derived from the published table	measured
low ÷ medium	0.258	0.112
high ÷ medium	3.94	4.00

High was close enough. Low cost us less than half of what we had assumed, so we had been pricing our cheapest tier as if it were 2.3 times more expensive than it actually is.

The thing that convinced me the measurement was real and not noise was that the ratios held across resolutions:

low  ÷ medium:  0.112, 0.112, 0.111
high ÷ medium:  3.998, 3.999, 3.999

Three different resolutions, same ratio to three decimal places. That is a parameter, not a sample. If those numbers had come back 0.11, 0.19, 0.14 I would have assumed measurement error and kept the published figure.

That is the check I would suggest to anybody doing this. A cost ratio that is stable across conditions you varied is a real multiplier you can build a price on. One that moves around is either noise or evidence that your model has the wrong shape.

A new model version at identical prices can still cost a quarter as much

The second one is stranger, and we caught it before it shipped rather than after.

The provider released a new generation of the same image model. Same price per token on every axis: image input, cached input, text input, output, all identical to the previous generation. Reading the pricing page, there is no change at all.

But the quality tiers went from three named levels to five, and a third-party benchmark on a fixed 2048×1152 image reported:

	output tokens
old model, `high`	1,413
new model, `high`	367

Same tier name, same price per token, about a quarter of the tokens.

I could not reproduce that directly, because at the time we had not integrated the new model. What made me believe it was that it agreed with a number we had measured ourselves for a completely different reason. Our own internal figure was high ÷ medium = 4.00 on the old model. 1,413 ÷ 3.85 lands on 367. So the third-party claim reduces to "the new model's high is roughly the old model's medium", and that is a statement our own independent measurement supports.

Worth being explicit about that reasoning, because it is the only reason I trusted a number I had not produced. A third-party benchmark you cannot reproduce is worth something if it agrees with a ratio you measured yourself for unrelated reasons. It is worth nothing on its own.

The provider did document this, sort of. Their model page says the five quality levels are not direct equivalents of the previous model's levels, and recommends evaluating with your own prompts. That sentence reads like standard hedging. It is actually a spec change with a number attached, and the number is not in it.

Why it matters more if you resell

If you are the end user of the API this is a pleasant surprise. You get the same work for less money, or you move up a tier.

If you resell, it is a trap, and the trap is shaped like a one-line config change.

Our users pick a quality tier and pay accordingly. High costs them four times what Medium costs. That multiple exists because High really did consume about four times the compute.

Swap the model ID to the new generation and nothing else, and the same button still sends quality=high, still charges four times, and now comes back with roughly what Medium used to produce. Nobody sees an error. The image still looks fine. The only thing that changed is that the customer is paying a 4× premium for something that is no longer 4× the work.

That is not a performance regression you can file and get to later. The thing you sold is not the thing you delivered. We ended up leaving the old model on its own row, untouched, and adding the new ones as separate options with their own tiers, measured separately, specifically so that nobody could make that change by editing one field.

The smaller one that nearly got us

Unrelated to models, same category of mistake.

We had a cost baseline that turned out to be wrong, $0.0021 where it should have been $0.00233, and three derived figures computed from it sitting in the same document. Fixing the baseline was a two-character edit. The three derived numbers stayed exactly where they were, looking correct, being wrong. They needed to go 4.7 / 2.4 / 2.2 to 4.3 / 2.1 / 1.9 and they were not going to do that on their own.

Derived numbers do not update themselves. They just keep being confidently wrong in a table somebody will read next quarter.

What we do now

We do not build a cost model on a vendor's published example table. It is illustrative, it is often derived from one configuration, and in our case the ratios taken from it were off by 2.3× on a tier we actually sell. Our own usage records are the only source that describes what we are actually being charged for.

Measure ratios, not absolutes, and vary something while you do it. A multiplier that holds steady across three resolutions is something you can price against. An absolute cost per image is out of date the next time anything changes.

Treat "the tiers are not direct equivalents" as a breaking change, not a disclaimer. Vendors write that sentence in the voice of a footnote.

And if you resell, put a measurement step between "new model is available" and "new model is live", because the config change that skips it is one line and reads as harmless.

We do this for fontvibe.ai, which is where the numbers came from. The reasoning is not specific to images or to any one provider.

Image prompting lessons: stop describing the image, hand the model a reference sheet

jack — Thu, 24 Sep 2026 16:10:01 +0000

Thirteen rounds, one afternoon, one word. The word was ONLINE and I wanted it drawn in a particular chunky cartoon style. I have the whole log because I was writing down what changed each round, which turned out to be the only reason I can now say anything useful about it.

Short version up front, since it isn't a twist: no amount of rewording got me there. I ended up drawing a reference sheet by hand and pointing at it.

I picked letters for this on purpose. Most image work gets judged on taste and those arguments go nowhere, but "is that an R" has an answer, and so does "is that corner curved". You can be wrong in a way somebody can check.

The scorer was happy the entire time

There was an aesthetic scorer in the loop. Every round it came back somewhere between 8.5 and 9.0. My own rating of the same images started around 3 and ended around 9.

Round 1, scored 8.5:

I had asked for small torn notches at the tips of the strokes. This has cracks across the entire body of every letter, plus a gloss I never mentioned.

The scorer was not malfunctioning. It is a decent-looking image. It answers "is this good" and I was asking "is this the thing I described", and once your model is competent enough to produce non-garbage those two questions stop being related. If I had let the score gate anything I would have stopped at round 1 and shipped a picture of broken glass.

Rounds 1 to 3 were aimed at something that was not there

Here is the part I am least pleased about.

I had looked at my reference at normal size, as a whole word, and read it as rounded letters with torn spikes poking out. So I spent three rounds describing torn spikes.

There were no spikes. When I finally cropped out single letters and blew them up, the letterforms were polygons made of straight facet cuts, and the things I had read as spikes were just the corners where two straight edges met. Nothing in that reference was torn, ripped, or broken. Three rounds of careful prompt surgery, aimed at reproducing a defect in my own eyesight.

Zoom in on one element before you write a single word about it. Whole-word at thumbnail size gives you the vibe and lies to you about the geometry.

The word you picked is arguing with you

This is the expensive one, and it happened three separate times with three different words.

I wrote	I meant	I got
`chamfer`	a flat angled corner	a 3D bevel, side faces lit and everything
`puzzle-piece-like`	letters sitting tightly together	jigsaw tabs. Actual ones.
`paper cutout`	flat, no depth	flat, with fold creases

That is round 10. I got precisely what I typed and nothing I meant.

The part that cost me rounds is that you can't fix this by pushing harder in the other direction. By round 8 the prompt carried "no 3D", "no bevel", "no side faces", "no internal lines", and the extrusion came back anyway, because chamfer and cut both point hard at "something was sliced here, show the marks". A pile of denials does not outvote the word itself. The word has to go.

So when a constraint keeps losing, stop reinforcing the constraint. Go find what it is losing to.

Say what it is, not how it got made

Two phrasings that mean the same thing to a person:

corners cut at irregular angles             # how it got made

the outer silhouette of every letter is an
irregular polygon, straight edges meeting
at sharp corners of varying length          # what it is

The first kept giving me cut marks. Seams inside the letter body, creases, and the little lit faces you would see if something really had been sliced through. The second gave me the shape and stopped.

My guess is that a verb names an event, and pictures of events come with the mess the event leaves behind. I would not push that too hard as a theory. As a habit it has held up: when junk keeps appearing inside the subject that you never asked for, go read your prompt for verbs.

Words about the room change the letters

The prompt is split into a subject half and a scene half. I had assumed that split meant something.

Material vocabulary in the scene half, describing the room, leaks onto the subject every time:

edge, core, depth used about the environment, and the flat letters grow 3D extrusion
voxel or cube, and the letter edges go blocky and stair-stepped
plastic, toy, brick, and the letters soften and round off

None of those were about the text. There's no compartment in there for "this clause is scenery". I ended up banning material words from the scene half entirely, objects and space and light only, and the drift stopped.

Giving up and drawing it

By round 11 the shape of the problem was obvious. Every fix moved something else. Tighten the silhouette, gloss comes back. Kill the gloss, the shape goes wrong. Fix the shape, a metaphor drags in creases. I was not converging, I was circling.

There is also an embarrassing middle stretch I have skipped over, where I landed a version I liked at round 5, showed it, and was told round 4 had been closer. Rounds 6 through 8 were me trying to get back to something I had already had.

So I drew the thing instead: a full A-Z and 0-9 sheet in the target style, one image. The prompt shrank to a pointer, roughly "follow the letterform skeleton in the reference, make the surface cleaner than the reference."

Eleven rounds of problems went away in one go. Not gradually. The 3D, the seams, the gloss, the jigsaw tabs, the creases, all of it, in a single generation.

Yes, drawing the sheet was work. It was less work than eleven rounds, and I still have the sheet.

One thing the reference could not do

It locked the shape, which also meant I could not change anything.

The last round wanted that exact shape with a glossier finish. Asked for in one sentence, the gloss dragged the 3D back in, same as round 8. Split across two sentences it behaved:

The 2D silhouette of every letter must stay identical to the reference.
No change to the letterform shape itself, no extrusion, no added bevel geometry.

Within that unchanged shape, push the surface material to a glossy,
high-shine polished enamel finish.

Identical information. Two sentences. I don't have an explanation for why that matters and I am slightly suspicious of it, but it has reproduced every time I h
ave needed to hold one property and move another, so I keep doing it.

If you take one thing from this: text is fine for a couple of constraints and it falls apart once you need five of them true at the same time. That is not a prompting skill issue, it's the channel. When you notice yourself circling, the question to ask is whether the thing you are describing could just be handed over instead.

This came out of building typeface generators, which live at fontvibe.ai. Nothing above needs them.

Unicode normalization is not a deduplication key

jack — Tue, 22 Sep 2026 17:29:47 +0000

I had 84,000 kaomoji to deduplicate and a key that looked obviously correct:

key = "".join(unicodedata.normalize("NFKC", text).split())

Strip the whitespace, normalize the Unicode, group by the result. Every
deduplication tutorial I have ever read says to normalize first. It took me one
afternoon of reviewing the output to realize that this key had quietly decided
(╥_╥) and (╥﹏╥) were the same record.

They are two different faces. The wavy mouth is a harder cry than the flat one.

NFKC folds four characters that matter here

NFKC is compatibility normalization. The K stands for compatibility, and the
point of it is to fold characters that exist in Unicode only for round-tripping
with older encodings: the fullwidth Latin letters, the halfwidth katakana, the
circled numbers, the ligatures. It maps them onto their "normal" equivalents so
that ｈｅｌｌｏ and hello compare equal.

That is correct when the text is prose and you want Ｅ－ｍａｉｌ to match
E-mail. It stops being correct once the characters are the content themselves.
Here is what it does to four characters that carry meaning in a kaomoji:

Codepoint	Name	NFKC result
U+FE4F	WAVY LOW LINE `﹏`	`_` (U+005F)
U+FF3C	FULLWIDTH REVERSE SOLIDUS `＼`	`\` (U+005C)
U+FF64	HALFWIDTH IDEOGRAPHIC COMMA `､`	`、` (U+3001)
U+3000	IDEOGRAPHIC SPACE	(U+0020)

The first one is the crying face. ﹏ is a wavy mouth and _ is a flat one, and
NFKC folds them together because one of them was originally a typesetting variant
of the other. The second one is a pair of arms. ＼(^o^)／ is a wide celebratory
shrug, \(^o^)/ is a narrow one, and after NFKC they are the same string.

The third one is a case where NFKC is right. A fullwidth and a halfwidth
ideographic comma really are the same punctuation mark, and (≖、≖╬) and
(≖､≖╬) really are one entry. That is what makes this hard. The normalization
is not simply wrong. It is right about maybe a third of what it touches, and you
can't tell which third without looking.

233 proposed merges, or 6,944

This is easy to measure on a published dataset, so you do not have to take my word
for it. The file below is the merged corpus, 82,109 entries, already deduplicated
on exact string match. Every collision found here is a proposed merge between
two entries that a human previously decided were different.

import gzip, json, unicodedata as ud, collections

def groups(keyfn):
    buckets = collections.defaultdict(set)
    for line in gzip.open("data/kaomoji.jsonl.gz", "rt", encoding="utf-8"):
        t = json.loads(line)["text"]
        buckets[keyfn(t)].add(t)
    g = [v for v in buckets.values() if len(v) > 1]
    return len(g), sum(len(v) for v in g)

strip_only = lambda t: "".join(t.split())
with_nfkc  = lambda t: "".join(ud.normalize("NFKC", t).split())

print(groups(strip_only))   # (233, 500)
print(groups(with_nfkc))    # (6944, 15038)

Stripping whitespace alone proposes 233 merges. Adding NFKC proposes 6,944,
covering 15,038 of the 82,109 entries. Auto-merging on that key deletes about
8,000 records, and you never find out which ones, because a merge leaves no
trace.

Some of what it finds is genuinely useful. These are real groups from that run:

(╥_╥)              (╥﹏╥)
(・∀・)ノ            (･∀･)ノ
(╯°□°)╯︵ ┻━┻       (╯°□°）╯︵ ┻━┻
(T_T)              (T＿T)          (Ｔ＿Ｔ)

The second row is a real duplicate. Halfwidth versus fullwidth katakana middle
dot is not something anyone chose, it is an artifact of which IME the author was
typing in. The third row is a table flip where one of the two parentheses came
out fullwidth, which is also an accident. The first row is two different faces.
The fourth row is three faces if you care about stroke weight and one face if you
don't.

There is no rule that separates those four rows. I tried several: character
class, width consistency within a single entry, whether the fold lands in the
mouth region. All of them got the first row wrong in one direction or the other.

Whitespace carries pose information

Stripping whitespace has the same problem on a smaller scale, because whitespace
is load-bearing in this data:

ʕ·ᴥ·ʔ          bear, facing you
ʕ·ᴥ·　ʔ         bear, looking left
ʕ　·ᴥ·ʔ         bear, looking right

( •_•)>⌐■-■    before the sunglasses
(•_•)>⌐■-■     also before the sunglasses, typed by someone else

The first three are three entries. The last two are one. Whether a given space is
a pose or a typo is not recoverable from the string, and I don't think it is
recoverable at all without knowing what the thing depicts.

Three keys, one of which is allowed to merge

key_exact = text                       # byte-identical; safe to auto-merge
key_loose = "".join(text.split())      # whitespace only; PROPOSES merges
key_nfkc  = "".join(ud.normalize("NFKC", text).split())   # output field only

key_exact collisions merge automatically with no review. key_loose collisions
go to a human queue, and 233 groups is an afternoon. key_nfkc is computed,
stored as a field in the dataset, and never consulted by the merge pipeline.
Consumers who want aggressive folding for search or for their own matching can
have it. It just doesn't get to delete anything.

That last key is the part I got wrong twice. My first instinct after it burned me
was to rip it out entirely, which would have been the opposite mistake, because
the folding is genuinely useful for recall. If somebody searches for (╥﹏╥) you
want the flat-mouthed one in the results too. So I kept computing it and took
away its ability to destroy a record.

Keep the queue small enough that you will actually read it

Normalization tells you which records might be the same. Whether they are the
same is a different question, and I am
not sure that one has an algorithmic answer.

So normalize to find candidates, keep the raw string as the only identity, and
put a human in front of anything that would destroy a record. If the human queue
is too big to review, the candidate key is too aggressive and needs tightening.
Nobody reviews 6,944 groups. They trust the key and move on, which is exactly how
the records disappear without anyone noticing.

The corpus is on GitHub with the raw JSONL and the verification scripts, and on
npm and PyPI as kaomoji-dataset if you just want the data:

npm i kaomoji-dataset
pip install kaomoji-dataset

https://github.com/Funovate/fontvibe-kaomoji

If you want to look at the entries rather than the code, the browsable version is
at fontvibe.ai/tools/kaomoji. Data is CC BY
4.0.

I merged the seven public kaomoji collections. They barely overlap.

jack — Tue, 22 Sep 2026 16:02:59 +0000

I wanted a complete list of kaomoji, the ( ͡° ͜ʖ ͡°) and ¯\_(ツ)_/¯ family. There isn't one.
So I merged the public collections instead, and the merge turned up something I
wasn't expecting: they hardly overlap at all.

Seven collections, 21 pairs, 0.3% median overlap

The ones I found:

fdw/rofimoji, the biggest English-side set
Allaman/emoji.nvim
aoguai/rime_kaomoji_dict
mtripg6666tdr/Kaomoji_proj
the Japanese IME dictionaries
kaosute
an older list of my own

Across the 21 pairwise combinations the median Jaccard overlap is 0.3%. Four
pairs share not a single entry. One pair clears 5%, rofimoji and emoji.nvim,
which are both English-side Linux pickers, so that one is no surprise.

The IME dictionaries are where most kaomoji actually live, and they share almost
nothing with any of the English sets.

Each entry in the merged file keeps the sources it came from, so you can check
this yourself:

import gzip, json, itertools, collections, statistics

by = collections.defaultdict(set)
for line in gzip.open("data/kaomoji.jsonl.gz", "rt", encoding="utf-8"):
    r = json.loads(line)
    for src in r["sources"]:
        by[src].add(r["text"])

o = [len(by[a] & by[b]) / len(by[a] | by[b]) for a, b in itertools.combinations(by, 2)]
print(statistics.median(o), max(o), sum(1 for x in o if x == 0))
# 0.003  0.079  4

None of the collections is close to complete, and after seeing those numbers I
understand why.

Coverage checked at run time

Saying "this has everything" is not worth much on its own, so
scripts/verify_coverage.py downloads each upstream file when you run it and
diffs it against the merged set. Nothing is precomputed.

Upstream	Entries	Covered	Missing
rofimoji	1,562	100.0%	0
emoji.nvim	2,016	99.9%	3
rime_kaomoji_dict	959	99.4%	6
Kaomoji_proj	2,166	97.8%	48

It prints all 57 misses, so you can look at them and disagree with me about what
counts as a kaomoji. Three rounds of that took rofimoji from 93.0% to 100.0%.

Labels, and where they fall down

Deduplicated it comes to 82,109 entries. Around 70k carry emotion, intent and
subject labels from a controlled vocabulary in en, ja, zh, es, pt and de, so
嬉しい, 开心, feliz and happy return the same entries instead of whichever English
word the original author happened to use.

Three things worth knowing before you use it:

The labels are coarse. 69,679 labelled entries share only 605 distinct label combinations, because they were applied in bundles rather than judged one at a time. Fine for pulling up a mood, not for finding one specific face.
About 15% is left unlabelled rather than guessed at.
Most of the corpus is 顔文字＋セリフ, a face plus a line of Japanese dialogue, which is not much use if you cannot read it.

That last one is why the English-language picker at
fontvibe.ai/tools/kaomoji shows 11,020, the
subset that renders safely outside Japanese contexts, while the full 82,109 sits
in the dataset and on the Japanese page.

Getting it

npm i kaomoji-dataset
pip install kaomoji-dataset

Both read a gzip off disk, so they are Node and Python only for now. They will
not bundle for a browser. The repo has the raw JSONL and a CSV as well:
github.com/Funovate/fontvibe-kaomoji

106 of the entries are not collected from anywhere. I drew them, for feelings
that had no kaomoji yet: 躺平, 社死, saudade, mamihlapinatapai. Those are CC0.

If you know of a collection I missed, say so and I will run the coverage script
against it.