DEV Community

Cover image for The Data Dictionary Should Live in the ERD, Not in a Spreadsheet
Son Tran
Son Tran

Posted on • Originally published at schemity.com

The Data Dictionary Should Live in the ERD, Not in a Spreadsheet

Disclosure: I build Schemity, a desktop ERD tool - this post is from our blog and uses it for the examples.

TL;DR: The meaning of a schema - what tables are for, who owns them, why a grouping exists - usually lives in heads, spreadsheets, or wikis that drift out of date. Schemity attaches markdown descriptions to entities, legends, and context views, shown behind a file icon on the diagram itself, so the data dictionary lives inside the ERD and stays where the schema is.

You can write what a table, a group of tables, or an entire view of your schema means directly into the diagram: Schemity stores a markdown description on entities, legends, and context views, one click away behind a file icon on the canvas. The data dictionary stops being a second document you maintain next to the ERD - it becomes part of the ERD.

That matters because the second document is where schema knowledge goes to die.

Where does the meaning of your schema actually live?

An ERD tells you what the tables are. It does not tell you what they are for - which team owns settlements, why orders_v2 exists alongside orders, what "T1" means in your warehouse naming. That knowledge has to live somewhere, and in most teams it lives in the worst possible places: in people's heads, in a spreadsheet someone started once, or in a wiki page nobody has opened since the quarter it was written.

The data team behind that linked piece puts it bluntly: most teams "grit their teeth and try and keep it all in their heads." One of their analysts describes coming back from a vacation to find 15 new tables in the warehouse with no indication of what they were for or who owned them - on a team of seven analysts, every one of those tables becomes a question someone has to answer in Slack, again and again. And tools like dbdoc exist precisely because of the chaos: its README opens with the warning that your team "will be living in chaos" until a data dictionary exists.

Why separate data dictionaries go stale

The traditional answers all share one flaw: they create a second artifact that must be manually kept in sync with the schema.

A spreadsheet dictionary needs a human to remember it exists every time a migration runs. A wiki page is even further from the schema - different tool, different login, different search. Database COMMENT ON statements are the closest option, but they are painful to write, format, and review, which is why they are so rarely maintained. Every one of these lives apart from the place where people actually look at the schema: the diagram.

Two artifacts, one truth, manual synchronization - that arrangement has a failure rate of exactly 100% given enough time. The dictionary does not die loudly; it just quietly stops being trusted.

Where should I document what my database tables mean?

As close to the schema itself as it can possibly live - and the closest place is the diagram your team already opens to understand the database.

In Schemity, entities and legends carry a markdown description. When the description is not empty, a file icon appears at the bottom right of the entity or legend on the canvas; clicking it opens a modal that renders the markdown.

File icon at the bottom right of an entity in Schemity, indicating the certificates table has a description

Modal rendering the markdown description of the certificates entity: purpose, when rows are created, and caveats A legend already names and colors a region of your schema - "Billing", "Owned by the payments team" - and its description is where the paragraph-sized knowledge goes: why the grouping exists, which service writes to it, the runbook link, the caveat a reviewer needs before touching anything inside it.

Where the descriptions are stored is a deliberate design choice. Schemity saves them in the workspace's JSON files and reads them back from there - it does not write them into the database as table comments. Storing documentation in the database means every description edit becomes a SQL migration: a schema change to review, apply, and possibly get approved, just to fix a sentence. That is scary and unnecessary for what is purely a documentation edit. It is also not portable - SQLite has no comment support at all, and the engines that do support comments each implement them differently. In the JSON, editing a description is exactly as safe as editing a text file, because that is all it is.

Context views carry descriptions too. A context view is a saved, focused view of your schema that shows only a subset of entities and the relationships between them. When a view has a description, its row in the Context Views list shows a file icon button that opens the same markdown modal - so the view that shows a bounded context can also say what that context is, who its audience is, and what "done" looks like for that slice of the schema.

Schemity Context Views list with a file icon button on the Auth view row, opening its markdown description

The diagram is the right home for the dictionary

Spreadsheet or wiki dictionary Markdown descriptions in the ERD
Where it lives A separate document, separate tool On the entity, legend, or context view itself
When the schema changes Someone must remember the second update You edit it where you made the change
How it is found Search the wiki, ask in Slack File icon on the diagram you already have open
How it is versioned Its own history, if any With the workspace's JSON files, in Git

The reason this works is the same reason context views work: the ERD single source of truth. There is one set of entities, and every view, legend, and now every description attaches to that one set - there is no second copy of anything to keep in sync. The descriptions live in the same plain JSON files as the rest of the workspace, so a desktop ERD tool that keeps its diagrams in your Git repo versions your data dictionary in the same commit that changes the schema.

This is the same principle as surfacing check constraints in the diagram: the ERD should carry the business meaning of the schema, not just its shape. Structure without meaning is what made those 15 mystery tables a problem. An offline ERD tool that holds both - the shape on the canvas, the meaning one click behind it - is the version of the data dictionary that actually survives contact with a moving schema.

Top comments (0)