DEV Community: Kuroshio Data

Parsing Japan's EDINET: UTF-16 CSVs, tabs inside fields, and semantics hidden in the element ID

Kuroshio Data — Thu, 06 Aug 2026 09:56:47 +0000

Japan has a full securities disclosure system, EDINET, run by the Financial Services Agency. It is the structural equivalent of EDGAR: every listed company files there, the filings are public, and since v2 there is a documented REST API with free API keys. On paper it should be as easy to build on as EDGAR.

In practice, English-language coverage of Japanese corporate events runs days to weeks behind, and for small caps it often never arrives at all. Having spent a while pulling this API apart, I don't think the reason is access. It's that the payload fights you in three specific ways, and only one of them is "it's in Japanese."

Getting a document at all

Two endpoints matter. documents.json?date=YYYY-MM-DD&type=2 lists everything disclosed on one day with metadata. Then documents/{docID}?type=5 returns that filing — as a ZIP archive. Inside the archive, alongside the XBRL, there's a CSV that is far easier to work with than the raw XBRL tree.

That CSV has nine columns: element ID, item name, context ID, relative fiscal year, consolidated/individual, period or instant, unit ID, unit, and value. So far, reasonable.

Then you decode it and get mojibake, because the file is UTF-16LE, not UTF-8. That one is quick to spot and quick to fix. The next one is not.

The bug I shipped

The file is tab-separated, so the obvious parser is split('\n') then split('\t'). I wrote that, tested it against several filings, saw sensible output, and shipped it.

It is wrong, and it fails silently.

Values in the last column are quoted, and inside those quotes EDINET happily includes raw tab characters and raw newlines. This is not a corner case I constructed: it shows up in the body text of extraordinary reports, where the filer has pasted a table describing a business transfer, and it can show up in the "purpose of holding" field of large-shareholding reports.

The failure mode is the nasty kind. A stray tab inside a quoted value shifts every subsequent column by one, so value reads out of the unit column and comes back empty. A stray newline splits one logical record into two malformed ones. You don't get an exception. You get a filing that parses cleanly and quietly reports nothing where the interesting text should have been. If you're eyeballing a sample of output, everything looks fine — because the records that break are precisely the wordy, unusual ones you're least likely to have in your sample.

The fix is the boring one: scan character by character, track whether you're inside quotes, handle "" as an escaped quote, and only treat a tab or newline as a delimiter when you're outside. About thirty lines. The lesson I'd actually pass on is not "write a real parser" — everyone knows that — it's that a delimiter-based parser on unfamiliar data should be treated as unvalidated until you've deliberately hunted for the longest, ugliest free-text field in the corpus and confirmed it survives.

The part where EDINET is genuinely good

Now the payoff, and it's a real one.

Japanese listed companies file an extraordinary report (臨時報告書) when something material happens: an M&A decision, a change of major shareholders or parent company, a change of representative directors, shareholders-meeting resolutions. It's the rough analogue of an 8-K.

The obvious way to classify these would be to run the Japanese body text through a model and hope. You don't have to. In EDINET's XBRL taxonomy, the element ID that carries the report body already identifies the event type. The body of a share exchange decision arrives under DecisionOnShareExchangeTextBlock. A change of major shareholders arrives under ChangesInMajorShareholderTextBlock. Voting results arrive under ResolutionOfShareholdersMeetingTextBlock.

These map one-to-one onto the items of Article 19(2) of the Cabinet Office Ordinance — the legal list of things that trigger the filing in the first place. The regulation's structure is carried through into the taxonomy, so classification is a dictionary lookup and it is exact. No NLP, no confidence score, no drift.

For a sense of what actually flows through: across 94 corporate extraordinary reports I measured over six days in late July 2026, the most common events were 23 shareholders-meeting resolutions, 19 stock option issuances, 17 parent-or-subsidiary changes, 18 significant-financial-event disclosures (impairments, debt waivers, special losses), and 7 major shareholder changes, with M&A decisions — share exchanges, business transfers, subsidiary acquisitions, splits — making up a long tail of one to three each.

Those numbers sum to more than 94 on purpose. One report can carry several event elements at once, so a single filing shows up under multiple categories. If you're aggregating, count filings and events separately or you'll double-count your way into a phantom M&A wave.

Volume runs roughly 15–50 per business day, spiking past 150 in late June when AGM voting results land all at once.

The English-name asymmetry

Here's a structural quirk that took me a while to understand, and that I think is the real reason naive translation pipelines produce garbage here.

EDINET knows the official English name of every entity registered as a filer. So for extraordinary reports, where the company itself is the filer, you get its real registered English name — not a machine translation, the name the company itself uses.

But large-shareholding reports (Japan's 5% rule, the 13D/13G analogue) are filed by the holder, not the issuer. So you get the holder's English name, and the company being accumulated appears only as Japanese text plus a securities code. There is no reliable English name for it in the filing.

The temptation is to machine-translate the Japanese company name. Don't. Japanese corporate names are full of traps — the same characters have multiple valid readings, and the company's own chosen romanization frequently isn't any of them. The securities code is stable, unambiguous, and the thing you should be joining on anyway. Emitting the Japanese name plus the code is the honest output; inventing an English name is how you end up confidently wrong about which company just got a new 8% holder.

What the 5% filings actually tell you

One detail worth knowing if you ever look at these. Under Japan's rules, a holder crossing 5% must state the purpose of holding, in prose. Buried in that prose is a specific legal term, 重要提案行為 — "acts of important proposal." It isn't decorative. Declaring it changes which filing regime the holder is under, because it signals intent to push management on things like board composition or capital policy.

Which means the single most useful activist signal in the corpus is a legal phrase, not a sentiment. You can match on it exactly. A fund you've never heard of that declares it is worth more attention than a famous name that files a routine passive-investment purpose.

Versus EDGAR

For contrast, the U.S. side is easier in every mechanical way. SEC EDGAR needs no API key and no registration, the data is public domain, daily indexes are plain text, and Form 4 ownership documents are clean XML. The tradeoffs are volume — a busy day is 500 to 2,000 Form 4s — and that the interesting bit is a flag rather than prose: whether a trade ran under a pre-scheduled Rule 10b5-1 plan, which separates routine sales from discretionary ones.

Japan gives you fewer filings with more semantics baked into the schema. The U.S. gives you more filings with cleaner mechanics. Both are free and official, and neither is what most people building "financial data" products are actually scraping.

Everything above is doable with the official API and a free key; nothing here needs a vendor. I do also run hosted versions that emit this as English JSON — Japan 5% / activist filings, Japan corporate events, SEC Form 4 — but the parsing notes are the part I'd have wanted to read six weeks ago.

— kuroshio-data

Tracking Japanese Activist Investors Without Reading Japanese: 5% Filings from EDINET as English JSON

Kuroshio Data — Tue, 04 Aug 2026 07:31:34 +0000

When Elliott built its position in Dai Nippon Printing, the disclosure was public within days. When Oasis went after Fujitec, same thing. When one of the Murakami-affiliated vehicles like City Index Eleventh starts accumulating a mid-cap, the paper trail is right there, filed under Japan's large shareholding rule — the local equivalent of a SC 13D/13G, triggered when anyone crosses 5% of a listed company, with amendments required for every 1% move after that.

The catch is where "right there" is. These filings live on EDINET, Japan's EDGAR. EDINET has an API, which is genuinely good news, but the documents themselves are Japanese-language XBRL and CSV wrapped in ZIP archives. The filer's name is in Japanese. The purpose of the holding — the field that tells you whether this is a passive index adjustment or someone planning to show up at the AGM with proposals — is a free-form Japanese paragraph. There is no English feed. If you don't read Japanese, you find out about a 5% crossing when Bloomberg or a fund letter mentions it, which can be weeks later, or never for smaller names.

I read EDINET filings anyway (I'm a Japanese solo developer, and I was already pulling the API for my own screens), so I automated the part that's annoying even for me: fetching, unpacking the XBRL, normalizing names to English, and classifying the purpose text. The result is an Apify actor called Japan Activist & 5% Filings Tracker. It's paid — $0.05 per filing, pay-per-result, more on that below — and it turns each large shareholding report into a record like this:

{
  "docId": "S100XXQ7",
  "submittedAt": "2026-08-03 09:12",
  "filingType": "amendment",
  "holderNameEn": "Sompo Holdings, Inc.",
  "holderNameJa": "SOMPOホールディングス株式会社",
  "issuerNameJa": "株式会社ティアフォー",
  "issuerSecCode": "XXXX",
  "holdingRatioPct": 17.49,
  "previousRatioPct": 15.82,
  "ratioChangePct": 1.67,
  "crossedThreshold": false,
  "jointHolders": 1,
  "isPotentialActivist": false,
  "activistSignal": "none",
  "purposeEn": "pure_investment",
  "purposeJa": "純投資（政策投資を含む）",
  "edinetDocId": "S100XXQ7",
  "source": "EDINET"
}

That one is boring on purpose: an insurer topping up an existing stake, crossedThreshold: false because it was already past 5%. The interesting rows are the ones where crossedThreshold flips to true, or where isPotentialActivist is set. That flag fires two ways. First, a maintained list of funds with an activist track record in Japan — Oasis, Effissimo, Elliott, the Murakami-affiliated entities, Strategic Capital, and so on, matched against the holder name even when they file through subsidiaries. Second, the purpose text itself: if the stated purpose signals intent to make "important proposals" to management (a specific legal phrase in these filings, because it changes the filing regime the holder is under), that gets flagged regardless of who the filer is. That second path is how you catch a fund you've never heard of before it has a reputation.

Running it

There's nothing to configure for the common case. Open the actor on Apify, hit Run with an empty input, and it fetches the latest large shareholding filings and writes them to a dataset you can read as JSON, CSV, or via the Apify API. To make it a feed rather than a one-off, add a Schedule in the Apify console — daily after Tokyo close works well, since EDINET filings cluster during Japanese business hours. Filers have five business days to disclose a crossing, so daily polling loses you nothing. From there it's a normal dataset: pipe it into a Slack webhook, a spreadsheet, or an LLM agent that reads purposeEn and holdingRatioPct and decides whether to wake you up.

What it costs, and where it falls short

Pricing is pay-per-result: $0.05 per filing returned, on top of Apify's small compute charge. A typical day in Japan produces a handful to a few dozen large shareholding reports, so a daily schedule usually lands in the range of a few dollars a month. No subscription, no minimum. If a run returns nothing new, you pay for nothing new.

Two honest limitations, both of which I hit while building it. The purpose field in the source is free-form Japanese prose, so purposeEn is a best-effort category (pure_investment, management_involvement, and a few others) plus the raw Japanese string — for edge cases you'll want the raw text and a translator. And the issuer side stays in Japanese: EDINET gives you the holder's English name but not a reliable English name for the company being bought, so the actor returns issuerNameJa plus the securities code rather than inventing an English name that might be wrong. The securities code is always there, which is what you should join on anyway.

If extraordinary reports are more your thing — Japan's rough 8-K equivalent, covering M&A decisions, board and major shareholder changes, AGM voting results — the companion Japan Corporate Events Monitor does the same treatment for those at $0.03 per filing, and there's a SEC Form 4 insider trades tracker if you want the US side from the same shape of pipeline.

Built and maintained by a solo dev. If something in the output looks wrong, the issue tracker on the actor page reaches me directly.

— kuroshio-data