Scoring English word rarity by frequency alone squashes advanced words into one tier: a test with 70 words from our production API

Hi, I'm Hatsuki. I'm building WordForge, a browser game where you build English words from letter tiles and battle with them as cards.

Every word you play gets one of six rarity tiers: N / C / R / SR / UR / LG. Rarer words score more, from 1x for N up to 8x for LG. The lookup checks an inflection table first, then a CEFR level list, then word frequency (a Zipf score).

This post isn't about how that pipeline works. It answers one question: what happens if you skip the CEFR list and rank difficulty by frequency alone? I ran 70 real words through the production API to find out. If you're building a word game or a learning app and want to estimate word difficulty automatically, I hope this helps.

What I did

• Hand-picked 70 English words, from very basic to very advanced (roughly CEFR A1 to C2). The picks are my own.
• Sent each word to the production dictionary endpoint POST /api/dictionary/lookup and recorded the rarity the game actually assigns, plus the frequency (Zipf score).
• Ran the same Zipf scores through a frequency-only rule to get a hypothetical rarity.
• Compared the two.

curl -s -X POST https://word-forge.replit.app/api/dictionary/lookup \
  -H 'content-type: application/json' \
  -d '{"word":"ubiquitous","direction":"en-ja"}'
# => {"definitionJa":"至る所にある","frequency":3.42,"rarity":"super_rare", ...}

(definitionJa is the Japanese gloss, because the game is aimed at learners in Japan.)

The frequency-only rule:

Zipf	≥ 5	≥ 4	≥ 2.5	≥ 1.35	≥ 1.12	below
Rarity	N	C	R	SR	UR	LG

A Zipf score of 0 (no data) maps to N, and words of 3 letters or fewer are capped at SR.

// Frequency-only baseline used for the comparison
function rarityByFrequency(word, zipf) {
  if (zipf === 0) return "N"; // no data: likely a typo or a made-up word
  let r =
    zipf >= 5 ? "N" :
    zipf >= 4 ? "C" :
    zipf >= 2.5 ? "R" :
    zipf >= 1.35 ? "SR" :
    zipf >= 1.12 ? "UR" : "LG";
  if (word.length <= 3 && (r === "UR" || r === "LG")) r = "SR";
  return r;
}

Result: 42 of 70 words changed tier

	Words
Same tier in both	28
Actual tier higher (pushed up by the CEFR list, etc.)	41
Actual tier lower	1 (apple: C by frequency, N in the game)

With frequency alone, almost every advanced word lands in R (Zipf 2.5 to 4).

Word	Zipf	Frequency only	Actual
aesthetic	4.02	C	LG
quintessential	3.03	R	LG
juxtaposition	2.96	R	LG
sycophant	2.20	SR	LG
resilient	3.50	R	UR
ephemeral	2.99	R	UR
mundane	3.43	R	UR
ubiquitous	3.42	R	SR
integrity	4.22	C	SR
sustainable	4.19	C	R

Almost all of the upper-intermediate and advanced vocabulary sits inside the Zipf 2.5 to 4 band. So under a frequency-only rule you get lots of R and almost no SR, UR or LG. In game terms, knowing hard words doesn't pay off.

Going the other way, the only word that reached LG on frequency alone was defenestrate (to throw someone out of a window, Zipf 1.04). Words below Zipf 1.12 are so unusual that normal players basically never type them.

What I learned

1. Frequency measures exposure, not difficulty

aesthetic has a Zipf score of 4.02. You see it all the time on social media, but for a learner it's a hard word (around CEFR C2). integrity is the same story: common in real text, but most junior-high students don't know it.

For learner difficulty you need a second signal: the order in which learners actually acquire words. In WordForge, the CEFR level list plays that role. Frequency is only the fallback for words the list doesn't cover.

2. Zero-frequency words are tricky

petrichor (the smell after rain) has essentially no frequency data (0), so the rule makes it N. I treat "no data" as "probably a typo or a made-up word" and give it N, but that punishes words that are well known yet rare in corpora. I plan to fix these with a hand-curated override table.

3. The test also exposed translation-quality issues

4 of the 70 words came back with a bad Japanese gloss:

Word	Returned gloss (meaning)	What it should be
wry	泣く ("to cry")	皮肉っぽい ("dryly mocking")
gaunt	(empty)	やせこけた ("haggard, thin")
cynical	キュニコス派 ("the Cynic school")	皮肉な, 冷笑的な ("cynical")
lethargic	嗜眠 ("lethargy", medical noun)	無気力な ("sluggish, listless")

wry looks like it got confused with cry. Because people use this to learn, I'm fixing these by hand. A language-education researcher also told me the game is good practice for spelling and recall, but that learning new meanings needs good translations and example sentences. So adding example sentences and pronunciation is one of the goals of our crowdfunding campaign.

Takeaways

• If you rank difficulty by frequency alone, intermediate and advanced words collapse into the middle tier (41 of 70 words scored lower than they should).
• For learners, check a learning-order list (CEFR or similar) first, and use frequency only for words the list doesn't cover.
• Zero-frequency words and translation quality are hard to solve automatically. Plan for manual overrides.

Try it

WordForge runs in the browser with no install and no sign-up (the UI is mostly Japanese, but the words are English). Try a few words and see what rarity they get:

👉 https://word-forge.replit.app/?utm_source=devto&utm_medium=article&utm_campaign=dev01

I'm also preparing a crowdfunding campaign on CAMPFIRE, a Japanese crowdfunding platform (under review; it will run until mid-January 2027). The goal is a classroom edition with a teacher dashboard, offered to 30 English schools at ¥1M and to 100 public schools for one year at ¥5M.

Feedback and bug reports are very welcome: [e-mail] / X: @WordForgeJP

Note: the level-based modes in the game are WordForge's own estimates and are not affiliated with any testing organization.