DEV Community

Andrey M.
Andrey M.

Posted on

I stopped diffing my Go structs against the schema by hand. Now CI does it.

#go #backend #postgres #database

A while ago I wrote about a column that changed type 8 days after I mapped it. int NOT NULL became decimal(19,2) NULL, my hand-written struct still said int64, and nothing complained. The fix back then was to point a code generator at the live database and diff its output against what I had.

That worked. It also had a weak spot I didn't want to admit: I had to remember to run it. The generator lived in an IDE plug-in and talked to the database over JDBC. Great on my machine, useless in a pipeline. A check that depends on a human remembering it is a check that runs on good days.

So I rewrote the generator in Go.

One binary, no JVM 

go install github.com/panedrone/go-sdm/cmd/sdm@latest
Enter fullscreen mode Exit fullscreen mode

It's called go-sdm. One static binary, no runtime dependencies, PostgreSQL, MySQL and SQLite. Same idea as before: an XML manifest says what to generate, the live database says what it actually looks like, and the generator writes the models and DAO in Go.

The part that matters for this story is one command:

sdm val
Enter fullscreen mode Exit fullscreen mode

It generates everything in memory, compares it with the files in the repo, and exits 1 if anything differs:

out of date: internal/dbal/models/asset.go
Enter fullscreen mode Exit fullscreen mode

That's the whole trick. If someone runs a migration and nobody regenerates, the build goes red. Not three weeks later as "why is this value off by a few cents" — on the next push.

# GitHub Actions, roughly
- run: go install github.com/panedrone/go-sdm/cmd/sdm@latest
- run: sdm val
  env:
    SDM_PWD: ${{ secrets.DB_PASSWORD }}
Enter fullscreen mode Exit fullscreen mode

The honest caveat: sdm val needs a database to read. I don't parse DDL, on purpose — the database knows things the DDL doesn't show: views, the real column types of a query's result set, procedure parameters, ref-cursors. So CI needs a reachable database with the current schema: a staging one, or a container you migrate right before the check.

Starting from an existing database

The first post was about a DB-first rewrite: eighteen tables already there, nobody planning to redesign them. Writing the manifest by hand for that is exactly the kind of busywork that makes people skip the whole idea. So:

sdm xml -orm sqlx > start.xml
Enter fullscreen mode Exit fullscreen mode

reads every table and view and prints a starting manifest: a model per table and a DAO per table, named after it, with CRUD and the lookups by foreign key already in place:

<dto-class name="sqlx-Task" ref="tasks"/>

<dao-class name="TasksDao">
    <crud dto="sqlx-Task" table="tasks"/>
    <query-dto-list dto="sqlx-Task" method="getTasksByGroup(gID)" ref="tasks(g_id)"/>
</dao-class>
Enter fullscreen mode Exit fullscreen mode

Run it again after a new table appears and it prints only the new one — whatever is already in the manifest is skipped. -orm picks plain database/sql, sqlx or gorm for the models, and it's required on purpose: I'd rather type six characters than find out later that a default chose for me.

This week I pointed it at a Postgres database with 21 tables: one command for the manifest, one sdm gen, and the data layer compiled. Most of the time went into reading the output.

DAO as variables, not structs

One more thing I changed compared to the usual generated DAO. There's no repository struct to construct and pass around, and no interface to mock. Each DAO function is a package-level variable:

var GetTestRating = func(ctx context.Context, tID int32) (res float64, err error) { ... }
Enter fullscreen mode Exit fullscreen mode

A unit test swaps the one it needs and puts it back:

saved := dbal.GetTestRating
dbal.GetTestRating = func(ctx context.Context, tID int32) (float64, error) { return 4.5, nil }
t.Cleanup(func() { dbal.GetTestRating = saved })
Enter fullscreen mode Exit fullscreen mode

No mocking library, no database in unit tests. The database shows up exactly once — in sdm val, which is the one place where it's supposed to be the source of truth.

The point, again

Last time the conclusion was: stop hand-maintaining the mapping between a table and a struct, regenerate it from the table. This time it's the second half of that sentence: and make the build fail when you forget. A generator you have to remember to run is just a nicer way to hand-maintain things.

go-sdm is here: github.com/panedrone/go-sdm. A working example with a generated data layer: sdm_todolist_go_react_vue_npm_sqlite3.

Top comments (0)