DEV Community

Japan Open Data Lab
Japan Open Data Lab

Posted on Edited on

Extracting Japanese addresses and refusing to guess the municipality code (span F1 0.998)

TL;DR — We released ja-address-ner-v1, a two-stage component that extracts Japanese addresses from text, splits them into prefecture / municipality / town / street-number / building, and attaches the official municipality code via a dictionary linker. On a sealed test of 5,000 items it scores span F1 0.9983 — and across all 4,814 test addresses it attached a wrong code exactly zero times, because when the code cannot be determined uniquely, it answers "undetermined" instead of guessing. Trained only on fictional addresses. Apache 2.0, ONNX included, ~22.5 ms per item on CPU including linking.

  1. Two stages, two different failure budgets. Stage one is a token classifier: find the address, split it into components (component Micro F1 0.9965; prefecture 1.0000, municipality 0.9945, town area 0.9924, street number 0.9995, building 0.9985). Stage two is a dictionary linker that maps the municipality to its official code. Extraction can be near-perfect and the pipeline still worthless if stage two guesses — so stage two is built to abstain.
  2. The abstention is the headline number. On cases where a code could be determined, assignment accuracy was 0.9924 — and every error was an over-rejection, never a wrong code. On cases designed to be undeterminable (same-name town areas, nonexistent addresses), the correct-rejection rate was 1.0000. Total across the sealed test: 4,814 addresses, zero wrong assertions. In address data, a wrong code is far more expensive than no code.
  3. The caveat, as always: every address in training and testing is fictional, assembled from public registries and embedded in synthetic contexts. Real addresses arrive misspelled, abbreviated, and mid-sentence in colloquial text, and performance there will be lower. The sealed numbers describe well-formed address text, and we say so on the card.

How it fits together

from transformers import AutoModelForTokenClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("japan-opendata/ja-address-ner-v1")
model = AutoModelForTokenClassification.from_pretrained("japan-opendata/ja-address-ner-v1")
Enter fullscreen mode Exit fullscreen mode

The full flow — extract, decompose, then link to a municipality code — lives in the bundled reference implementation, examples/infer_address.py. One setup step matters: the dictionary is not bundled. You build it from public data (Japan Post's postal-code data and the Digital Agency's Address Base Registry) following linker/DICTIONARY-SETUP.md. That is deliberate: municipalities merge and split, and a replaceable dictionary lets the component track administrative changes without retraining the model.

The ONNX build (fp32) matched PyTorch exactly — address spans, components, and linker output — on 200 development and 200 sealed items. End-to-end speed, linker included: ~22.5 ms per item on a Core i7-11700F.

Measured results (sealed test, 5,000 items, single final run)

Stage 1 — extraction and decomposition:

  • Address span exact-match F1: 0.9983
  • Component decomposition Micro F1: 0.9965
  • False detections on 500 address-free sentences: 0

Stage 2 — dictionary linker:

  • Code assignment accuracy on determinable cases: 0.9924 (all errors were over-rejections)
  • Correct rejection on undeterminable cases: 1.0000
  • Wrong code assertions across all 4,814 addresses: 0

Training data

Entirely synthetic. Fictional addresses were generated from Japan Post postal data and the Digital Agency's Address Base Registry (both PDL 1.0) and embedded into contexts from NVIDIA's Nemotron-PII-Japan (CC BY 4.0) and template sentences. No real person's address appears anywhere in it. Splits: 100,000 train / 5,000 development / 5,000 sealed, checked for cross-split overlap.

What didn't work / Limitations

  • Synthetic addresses, well-formed text. The 0.9983 was measured on generated address sentences. Addresses as people actually write them — dropped components, old municipality names, notation variants, colloquial surroundings — will score lower, by an amount we have not measured and will not invent.
  • "Undetermined" is a feature you have to handle. Nonexistent addresses and genuinely ambiguous same-name areas come back without a code. If your pipeline treats a missing code as an error rather than a routing decision, this component will feel broken; it is doing what it was built to do.
  • Out of scope: assigning postal codes, and fully normalizing street-number notation variants. The linker gives you the municipality code; the rest of address hygiene is downstream work.
  • The dictionary is yours to build and maintain. No bundled dictionary means no stale dictionary shipped by us — and also means setup takes one extra step, following the public-data build guide.

This is the seventh in our line of small, single-purpose Japanese models, and the second built around a refusal: the document router abstains on type, this linker abstains on code. Components that know when not to answer are the ones you can safely chain.

If you process Japanese addresses today: where does your pipeline put the ambiguity — a master table, fuzzy matching, or a human with a search box?

Get the next release notes and our Kaggle experiment logs by email: https://jpopendata.jp/ (newsletter signup in the footer).

Top comments (0)