We keep a set of regulatory reference tables by hand. Fifty jurisdictions, one row each, describing what it takes to get a crypto licence in that country: the regulator, the operative instrument, the statutory minimum capital, the licence term, the timeline. The tables sit behind a comparison tool on our site and they change whenever a regulator moves.
People were screenshotting them. That is a reasonable thing for a reader to do and a bad thing for the reader to end up with. A screenshot has no date, no provenance and no way to check whether the number moved last Tuesday. So we published the underlying slice as a dataset with a DOI.
The interesting part was not the export. Writing a CSV is twenty lines. The interesting part was working out what metadata a dataset needs before anyone should be willing to cite it, given that the source is a living file we edit every week.
Here is what we ended up with, and why each piece is there.
Freeze the generator, not just the data
The obvious move is to record which version of the data you exported. That is not enough. The export is a function of two things: the input and the code that transforms it. If you record only the input, you cannot reproduce the output six months later, because the generator has moved on.
So the provenance block records both, by hash, and keeps a frozen copy of each:
"source": {
"matrix": "Website/data/crypto-licensing-matrix.json",
"matrix_sha256": "5ec145639c8c1f371759c396355d01653b0711075feb932df0843331d1852f6a",
"generator": "Pipelines/open-data/build.py",
"generator_sha256": "c81d9ad0242674c21f3f8ac547ca72cc0ea784a6322824f811f0bbf8116da4b5",
"git_commit": "0c541cb9654d37fb39bc5ce09404146cad228718",
"git_tree_dirty": false,
"extracted_on": "2026-09-05",
"matrix_snapshot": "...-2026-09-05.matrix.json",
"generator_snapshot": "...-2026-09-05.build.py"
}
The field that earns its place is git_tree_dirty. A commit hash tells you what was committed. It does not tell you whether the working tree had uncommitted edits when the export ran. If that flag is true, the commit hash is a lie about what produced the file, and anyone reading the metadata deserves to know that rather than to infer reproducibility that is not there.
Enumerate the holes
A blank cell in a CSV is ambiguous. It might mean not applicable, or unknown, or nobody has published a comparable figure, or a transcription gap. Readers resolve that ambiguity by guessing, and a surprising number of them guess zero.
We list every empty cell explicitly:
"empty_cells": [
{"slug": "dominica", "column": "year1_cost_band"},
{"slug": "romania", "column": "timeline_months_min"},
{"slug": "romania", "column": "timeline_months_max"}
]
Three cells out of one thousand and fifty. Small enough that we could have quietly filled them with a plausible number, which is exactly why we did not. The accompanying rule says what the list does and does not mean: it enumerates the gaps, it does not classify them, and you should read the linked source before relying on any one of them.
An inventory of holes is a stronger honesty signal than a completeness percentage, because it is checkable. A reader can go and look at those three cells.
Refuse to compute things that expire
The most requested column was a US dollar equivalent. We do not publish one.
A statutory capital requirement denominated in euros is a fact with no expiry. The same figure converted to dollars is a fact about an exchange rate on an unnamed day, silently baked into a file that will be downloaded for years. It looks like the same kind of data and it is not. If a statute names a dollar amount, that dollar amount stays, because it is statutory. Everything else stays in the currency the law uses.
The general rule we took from this: an export should not contain a value whose truth depends on a date the export does not record.
Two DOIs, and only one of them can live in the file
Zenodo mints two identifiers. A version DOI names one immutable release. A concept DOI names the dataset as a whole and always resolves to the newest version.
You want readers citing the version DOI, because a citation should point at bytes that cannot change under the reader. You want the landing page advertising the concept DOI, because that is the address that stays useful.
The version DOI goes inside the file. The concept DOI cannot, and the reason is structural rather than a matter of preference: Zenodo mints the concept DOI at first publication, and a versioned file is frozen after it is published, so a file published in release one cannot contain an identifier that did not exist when it was written. We record the concept DOI in a separate manifest and on the landing page, and the file itself says why it is not there.
The release label is not a verification date
This one took the longest to accept. We label releases by extraction date, and it is tempting to let readers treat that as "everything here was checked on that day". It was not. It means the values were what the source recorded on that date. When each individual figure was last verified against a regulator is a per-row fact, and it lives with the guide the row came from.
Compressing per-row verification history into one file-level date would be the single most misleading thing we could do, precisely because it would look like rigour.
What this actually is
The export itself is a small Python script. Almost all of the work was deciding what not to publish. The dataset is fifty records and twenty-one columns, released under CC BY 4.0, with a field dictionary that names every column and enumerates the permitted values for the categorical ones.
If you maintain reference data by hand and people cite it informally, the metadata above is most of what turns it into something citable formally. None of it requires a research infrastructure. It requires deciding, once, that a reader who checks your work should be able to.
The dataset and its field dictionary: https://jagelski.com/data/
DOI: https://doi.org/10.5281/zenodo.22334372
Top comments (0)