The problem: Japanese data has no single doorway
The first reflex for a Japanese company is to search the name and read whatever page comes back. That works for one company and collapses on the second: the pieces you need sit in four registers with four access rules, none of them a page you want to automate against. This is the map of which official source answers which question, and where each stops.
The identity layer: corporate numbers
Company names are not unique and get transcribed inconsistently, so a name match is not an identity. The corporate number (法人番号) is the stable key: a 13-digit identifier from the National Tax Agency, reachable through the corporate number API with a free application ID, plus a CSV publication for bulk work. Use it as the join key for everything below. The corporate number API: looking up any Japanese company.
Filings and financial statements: EDINET API v2
For anything financial, the official route is EDINET, the FSA's disclosure system. Filings arrive as XBRL, and API v2 gives you a subscription key, date-range search and a flattened CSV form that most people prefer once they try parsing raw taxonomies. The sharp edges matter: the CSV is UTF-16, tab-delimited and quotes every field, so fix encoding and delimiter up front or the pipeline mangles numbers silently. EDINET API v2: XBRL filings as CSV without the pain.
Statistics: e-Stat
Population, GDP, CPI, employment: the official numbers come from the Statistics Bureau through the e-Stat API with an application ID. The part that trips people up is the two-step flow — ask the metadata endpoint what tables and categories exist, then request the data for the table you picked. Skip the first step and you are guessing at identifiers. Getting Japanese government statistics via the e-Stat API.
Market data: an IRBANK-compatible endpoint
Equities are where the global free feeds fail you: US and European tickers are covered, Japanese ones are missing, and the coverage wall appears only after you have built on them. The IRBANK-compatible endpoint fills that role — daily OHLCV, valuation metrics, dividends, margin balance and segment disclosure through one code per company, with no scraping of rendered pages. IRBANK API: Japanese stock data without scraping.
The glue: where the application IDs live
Application IDs and subscription keys are credentials, so they belong where the calling code cannot read them. A Google Apps Script web app holds a key server-side and exposes a thin endpoint at zero cost, with one trap: use a deployment ID, not the /dev URL, or it only works for you (GAS web apps as API proxies). In my setup those IDs never sit in an agent's environment — they live in a credential store and arrive through a proxy, the pattern behind trustless. For lightweight record stores rather than acquisition, the same CLI access pays off (kintone from the CLI).
The shape they share
- Official first. An official feed has a stated contract and a maintainer paid to publish the numbers; scraping is the fallback.
- Identity before metrics. Resolve the entity key, then attach filings, statistics and prices. Merging without a stable key is how double counting starts.
- Read the format spec, not the sample. UTF-16 and tab delimiters look fine in a preview and break a loader.
- Treat rate limits as a schedule — they tell you the cadence the pipeline should run at.
None of these is turnkey, and each is documented better in Japanese than in English — a real cost when you work in English. The alternative is scrapers against four rendered sites, and that bill arrives every time a layout changes.
Which country's official data have you had to wire in for the same reason — and did an official API exist, or did you end up maintaining scrapers?
Originally published at agentjournal.dev.
Top comments (0)