DEV Community: Fred Santana

Surviving Midnight SDK: a 700-line cure for the silent failure problem

Fred Santana — Mon, 27 Apr 2026 23:09:46 +0000

Why I built midnight-doctor — a pre-flight check that catches the 16 hours of debugging you don't know you're about to spend

"The information you need to align your stack already exists. It's just not executable."*

After five months building four applications on Midnight Network, I've concluded the protocol isn't the problem. The protocol is genuinely good — Compact compiles ZK circuits in three seconds without a trusted setup ceremony, and the shielded/unshielded/dust split is the cleanest answer to compliance-grade privacy I've seen.

The problem is everything that surrounds it. Specifically, the unforgiving math of: a fast-moving SDK + multiple environment tracks + a single npm namespace + zero error messages when versions misalign.

This post is the story of one specific 6-hour debugging session, the pattern I extracted from it, and the ~700-line tool I built so the next person doesn't pay the same tax.

The 6-hour silent failure

DPO2U Wallet, March 2026. I bumped @midnight-ntwrk/wallet-sdk-facade to 2.0.0 because npm marked it as latest. My local stack ran midnightntwrk/midnight-node:0.21.0 (preprod-targeted). I wrote ~50 lines of wallet bootstrap, started the dev loop, and watched:

const wallet = await WalletFacade.init({
  configuration,
  shielded,
  unshielded,
  dust,
});

await wallet.waitForSyncedState();
console.log('synced!');

The synced! line never printed.

No error. No timeout. No log. The wallet just sat in syncing state forever. I added more logging — got more "syncing" lines. Restarted Docker. Wiped state. Tried a fresh seed. Re-cloned the repo. Asked Discord. Read the SDK changelog (which didn't exist for that version). Stared at the indexer logs for 40 minutes.

Six hours in, almost by accident, I noticed the symptom: subscribeRuntimeVersion was firing once, returning, and never firing again. The standalone node closes that subscription early. The WalletFacade.init({...}) constructor in 2.x wires sync to that subscription. Result: a single missed event silently kills the entire sync loop, with zero surface.

The fix was to downgrade to facade 1.0.0 (preprod track). Forty seconds of work, after six hours of debugging. The bug was real but the cost was information asymmetry: nothing in the documentation, npm metadata, or runtime told me that "this SDK version is incompatible with this node version."

That asymmetry is the real bug. Not the runtime subscription closing. Not the constructor wiring. The fact that the system has the information needed to diagnose itself, but doesn't bother.

The pattern: silent failures from missing cross-references

After that session, I started cataloguing other times I'd been bitten the same way. Within a week I had a list:

Symptom	Root cause	Time spent
`waitForSyncedState()` hangs forever	facade 2.x + node 0.21.0 mismatch	~6h
`npm install` returns ENOTFOUND	community tutorial said `.npmrc` should point at `npm.midnight.network` (domain doesn't exist)	~2h
Transactions silently fail to submit	Two versions of `@midnight-ntwrk/ledger-v7` in `node_modules` (transitive bump)	~3h
Indexer crash-loops on startup	`indexer-standalone:4.0.0-rc.4` requires a `subscription:` block in `indexer.yml`, undocumented	~1.5h
WalletFacade behaves differently on standalone vs preprod	Same SDK, different node behavior, no warning	~4h

Across five incidents: ~16 hours of debugging silent failures. Each one had a root cause that was, with the right cross-reference, detectable in seconds:

package.json says facade is 2.0
docker ps says node is 0.21.0
A lookup table says those two are incompatible
→ Print a clear error.

The system already has all three pieces of data. There's just no glue.

The tool: `midnight-doctor`

The fix isn't a new feature in the SDK. It isn't better docs (which decay). It's an executable cross-reference — a script that reads your project, your Docker stack, and your config files; matches them against a curated compatibility table; and tells you what's wrong.

I built midnight-doctor over a weekend. ~700 lines of Node, zero runtime dependencies, single-binary install:

npx midnight-doctor

Run against my own legacy midnight-hello-world repo, it produces:

── midnight-doctor ──
project: /root/midnight-hello-world

⚠ SDK track: Preprod 3.x (legacy, OK for existing apps)
   Detected from wallet-sdk-facade@2.0.0.
⚠ Track is deprecated
   SDK 3.2 / facade 2.0 is 2 majors behind. Current is wallet-sdk@1.0.0 /
   facade@4.0.0 (released 2026-04-23). The WalletFacade.init({...}) constructor
   used in 2.0 has been removed; 4.0 reverted to `new WalletFacade(s, u, d) +
   .start()`. Plan a migration.
⚠ WalletFacade.init() in 2.x stalls on standalone dev nodes
   In SDK 2.x, WalletFacade.init() subscribes to runtime version events that the
   standalone node closes early. The wallet hangs in 'syncing' state forever
   with no error.
   → fix: Either (a) develop against preprod once past hello-world, or
     (b) upgrade to wallet-sdk-facade@4.0.0 which reverted to constructor +
     .start() pattern.
✓ node: midnightntwrk/midnight-node:0.21.0
✓ indexer: midnightntwrk/indexer-standalone:4.0.0-rc.4
✓ proof-server: midnightntwrk/proof-server:7.0.0
✓ midnight-node:0.21.0 matches SDK track

summary: 4 ok  3 warn  0 error  0 info
Status: workable, but warnings deserve a look.

The 6-hour incident from March is now a 30-second output.

Architecture

The tool is structurally trivial. Three scanners, one diagnosis engine, one report formatter. Total surface:

midnight-doctor/
├── bin/midnight-doctor.js   # CLI entry, arg parsing, exit codes
├── lib/
│   ├── scan-package.js      # reads package.json + walks node_modules
│   ├── scan-docker.js       # runs `docker ps`, parses image tags
│   ├── scan-config.js       # reads .npmrc, indexer.yml
│   ├── diagnose.js          # cross-references findings with matrix
│   ├── report.js            # ANSI-colored pretty output
│   └── index.js             # public API
├── data/compatibility-matrix.json   # the source of truth
└── test/diagnose.test.js    # node:test, no framework

The compatibility matrix is the product

The ~700 lines of code are scaffolding. The actual value is the JSON document at data/compatibility-matrix.json. It encodes three things:

Tracks — known-good combinations of SDK + Docker images, labeled (current, preprod-3x, preprod-1x)
Known issues — specific bugs keyed by package version range or config pattern
Cross-cutting checks — rules that fire when two scanners disagree (e.g., node tag vs SDK track)

A snippet:

{
  "tracks": {
    "current": {
      "label": "Current (latest npm + preprod compatible)",
      "node": "0.21.0",
      "indexer": "4.0.0-rc.4",
      "proofServer": "7.0.0",
      "packages": {
        "@midnight-ntwrk/wallet-sdk": "1.0.0",
        "@midnight-ntwrk/wallet-sdk-facade": "4.0.0",
        "@midnight-ntwrk/compact-runtime": "0.15.0",
        ...
      }
    }
  },
  "knownIssues": [
    {
      "id": "facade-2x-init-bug",
      "severity": "warn",
      "match": { "type": "package-version",
                 "package": "@midnight-ntwrk/wallet-sdk-facade",
                 "range": "2.x" },
      "title": "WalletFacade.init() in 2.x stalls on standalone dev nodes",
      "fix": "Either develop against preprod, or upgrade to facade 4.0.0..."
    }
  ]
}

This file is the same information that lives, scattered, across Discord pinned messages and individual developers' heads. Centralizing it in a JSON document accomplishes three things:

Source of truth — there's now a place to point people instead of "search the channel"
Versioned — the file has a verifiedAt: "2026-04-27" field; staleness is detectable
Executable — code can act on it; humans can read it

The scanners are deliberately dumb

Each scanner is one file, one responsibility, ~50 lines.

// lib/scan-docker.js (excerpt)
export async function scanDocker() {
  const findings = { dockerAvailable: false, containers: {} };
  try {
    await exec('docker', ['version', '--format', '{{.Server.Version}}']);
    findings.dockerAvailable = true;
  } catch { return findings; }

  const { stdout } = await exec('docker', [
    'ps', '--format', '{{.Image}}|{{.Names}}|{{.Status}}',
  ]);

  for (const line of stdout.split('\n').filter(Boolean)) {
    const [image, name, status] = line.split('|');
    const [imageName, tag = 'latest'] = image.split(':');
    if (KNOWN_IMAGES[imageName]) {
      findings.containers[KNOWN_IMAGES[imageName]] = { image: imageName, tag, name, status };
    }
  }
  return findings;
}

No Docker SDK dependency, no parsing library — just child_process.execFile and String.prototype.split. If Docker isn't installed, the scanner returns { dockerAvailable: false } and the diagnosis engine emits an info diagnostic instead of crashing.

The package scanner is similar — walks node_modules recursively (capped at 5000 directories so monorepos don't hang), records every @midnight-ntwrk/* it finds and what version. Duplicates fall out as a side effect: if the same package name has more than one version in the array, it's a duplicate.

const allInstances = await findAllInstances(projectDir, MIDNIGHT_NS);
for (const [name, versions] of Object.entries(allInstances)) {
  const unique = [...new Set(versions)];
  if (unique.length > 1) findings.duplicates[name] = unique;
}

Three lines. That's the entire duplicate-detection logic. The remaining ~50 lines of the scanner are filesystem traversal — boring, mechanical, but it works on every Node project regardless of package manager (npm, pnpm, yarn, bun all create node_modules directories with the same shape).

The diagnosis engine is glorified table joins

lib/diagnose.js takes the three scan results plus the matrix and emits diagnostics. The pattern repeats per check:

function diagnoseCrossCutting(pkg, docker, matrix) {
  const facadeVersion = pkg.installed['@midnight-ntwrk/wallet-sdk-facade'];
  const nodeContainer = docker.containers.node;
  if (!facadeVersion || !nodeContainer) return [];

  const track = matrix.tracks[detectTrack(facadeVersion, matrix)];
  if (track.node && track.node !== nodeContainer.tag) {
    return [{
      id: 'node-track-mismatch',
      severity: 'error',
      title: `midnight-node:${nodeContainer.tag} doesn't match SDK track (expects ${track.node})`,
      detail: `Mixing causes silent sync failures with no error.`,
      fix: `Either update the node container to ${track.node}, or align SDK to a track that matches your node version.`,
    }];
  }
  return [{ id: 'node-track-match', severity: 'ok',
            title: `midnight-node:${nodeContainer.tag} matches SDK track` }];
}

That's the entire body of the cross-cut check that would have saved my 6-hour March incident. Twelve lines. The information was there the whole time.

The full diagnosis file is ~200 lines. Most of it is the same shape: pick two facts from the scans, compare against the matrix, emit a diagnostic. It's deliberately boring code.

What this isn't

Scope discipline is half the value of a tool like this. Things midnight-doctor deliberately doesn't do:

It doesn't auto-fix. No --fix flag yet. The cost of a wrong auto-fix in this domain (bricked node_modules, lost work) outweighs the convenience.
It doesn't curl health endpoints. It checks Docker container presence, not health. A separate health-check script (midnight-health-check.sh) covers that already in my infra repo.
It doesn't probe the chain. No getBlockNumber(), no balance lookup. Those depend on a working SDK, which is what doctor is meant to validate before you try to use it.
It isn't a package manager. It won't run npm dedupe for you. It tells you to run it.
It doesn't write code. It doesn't generate scaffolds, codemods, or migrations. That's a different tool (create-midnight-app, eventually).

Each non-feature is a deliberate choice to keep the surface small enough that the tool stays correct.

What I'd build next, if there's appetite

midnight-doctor is the smallest useful version. The compatibility matrix as data unlocks several adjacent tools:

create-midnight-app — scaffold a project where midnight-doctor is wired into npm install lifecycle. The matrix becomes the default versions.
A codemod for the 2.x → 4.x facade migration. The API change is mechanical: WalletFacade.init({...}) → new WalletFacade(...) + .start(...). AST transform.
Compactc compiler version check. Today it's via shell-out. With a parser, doctor could read your .compact source and say "this uses syntax X, requires compactc ≥ Y."
Remote-fetched matrix. Right now the matrix is bundled. A nightly job at https://compatibility.midnight.network/matrix.json would let users not need to bump the doctor itself.
Editor integration. A VSCode extension that runs doctor on save and surfaces diagnostics in the problems panel. The CLI is for CI; the editor is for dev loop.

The first one is the highest leverage by far — it short-circuits the entire onboarding problem. But it's a much bigger project. Doctor is a Trojan horse for the matrix; once the matrix exists and people accept it as canonical, the rest is plumbing.

The broader argument: tools beat docs

Documentation rots. A README written today describes a system that, six weeks from now, has moved two majors. Anyone who builds in a fast-moving ecosystem has lived this: the official doc says "use SDK 1.0", the npm latest tag says 4.0, the GitHub README says 3.0, and the Discord pinned message says "we know, sorry, the docs are stale." None of those sources is wrong. They were all true at some point. The problem is they don't update each other.

Executable knowledge updates itself. A JSON file with a verifiedAt date that's three weeks old is visibly stale. A doc that's three weeks old looks identical to a doc that's three days old. Tools force the question "is this still true?" in a way that prose doesn't.

This isn't unique to Midnight. Every fast-moving stack reaches the point where the gap between "what the docs claim" and "what actually works" is wide enough to swallow new developers. The fix is the same: take the lookup table out of human heads, encode it as data, ship it as a tool that runs in seconds.

The Midnight protocol team doesn't need to build midnight-doctor. They could. Anyone could. The information already exists — in pinned messages, in CHANGELOG.md files, in the heads of the half-dozen people on Discord who answer the same question every week. The work isn't producing the information. It's transcribing it once, into a format machines can act on, and committing to keeping it current.

That's what this ~700-line tool does. It's not clever. It's not hard. It's just nobody had done it.

If you've spent hours on a Midnight silent failure, please run npx midnight-doctor against your project before your next debugging session. If you find a bug doctor missed, open an issue — the matrix is the artifact, the code is just glue.

Repo: github.com/fredericosanntana/midnight-doctor
Install: npm install -g midnight-doctor or npx midnight-doctor
License: MIT

If this was useful, the people who actually maintain Midnight (@MidnightNtwrk) deserve the visibility — half this matrix came from their Discord answers. The other half came from getting burned. Both contributions are essential.

Appendix: the full check list

For reference, every check midnight-doctor currently runs:

Package scanner:

✓ Detect SDK track (current, preprod-3x, preprod-1x) from wallet-sdk-facade major
✓ Flag deprecated tracks with migration guidance
✓ Detect major-version mismatch across wallet-sdk-* subpackages
✓ Detect duplicate @midnight-ntwrk/ledger-v7 in node_modules
✓ Detect duplicate @midnight-ntwrk/compact-runtime in node_modules
✓ Flag wallet-sdk-facade@2.x standalone init bug

Docker scanner:

✓ Detect running midnight-node, indexer-standalone, proof-server
✓ Cross-reference node tag with SDK track

Config scanner:

✓ Flag .npmrc with bogus npm.midnight.network registry
✓ Flag indexer.yml missing subscription: block

Cross-cutting:

✓ Node tag ↔ SDK track consistency

Roadmap:

☐ Compact compiler version vs runtime
☐ Health probes (RPC, GraphQL, proof endpoints)
☐ Monorepo workspace iteration
☐ --fix mode for safe auto-corrections
☐ Remote-fetched matrix

Total checks today: 11. Adding more is one PR each.

How a Blockchain Transaction Works: From Start to Final Block

Fred Santana — Tue, 24 Mar 2026 15:16:32 +0000

Why should a compliance professional care about this?

I spend a good part of my days thinking about how to protect personal data. LGPD (Brazilian General Data Protection Law), privacy policies, legal bases — that's my world. When I started studying blockchain seriously, I realized that it wasn't enough to understand what the technology does. I needed to understand how information enters this network and why, once it's in, it's practically impossible to remove it.

This "practically impossible" is what keeps those who work with compliance awake at night. But before entering this debate — which will be the heart of this 30-day campaign — we need to understand the basic mechanics. How does a transaction originate, travel through the network, and become permanent?

I'll explain without a single line of code. I promise.

The analogy of the digital notary office
Think of a notary office. You go there, sign a document, the notary recognizes your signature, registers everything in a book, and stamps it with the date and time. From that moment on, that record officially exists.

A blockchain transaction follows a similar logic — only without the notary. There is no person or company at the center deciding what is valid. The entire network plays this role. And the "registry book" isn't kept in just one place: it's copied on thousands of computers around the world.

Does that seem excessive? It's precisely this excessiveness that makes things secure. And it's this same excessiveness that creates the challenge for the LGPD (Brazilian General Data Protection Law).

The five stages of a transaction

Creation — "I want to register something" It all starts with an intention. Someone wants to send a value, register data, or perform an action on the network. This intention becomes a "transaction" — a package of information that basically says: who is sending, to whom, and what.

Think of it like filling out a form at the registry office. You haven't registered anything yet — you've only written what you want to do.

Connection with compliance: even at this stage, if the data being registered is personal, the LGPD comes into play. Article 7 requires a legal basis for any processing of personal data. The simple creation of a transaction containing a person's data is already "processing" under the law.

Signature — "I confirm that it's me" Before submitting the form, you need to prove that it's you. In a physical notary's office, this would be your notarized signature. On the blockchain, there's a mechanism called a digital signature.

It works like this: you have two keys. One is like your safe's number (public — anyone can see it). The other is like the combination that opens the safe (private — only you know it). When you "sign" a transaction, you're using your private key to create a unique mark that anyone can verify using your public key, but that no one can forge.

It's as if your signature at the notary's office were impossible to copy. Literally impossible — not difficult, impossible.

Connection to compliance: the public key, although it doesn't contain your name, can be traced back to you under certain conditions. Article 12, §2 of the LGPD (Brazilian General Data Protection Law) states that data that can be used to identify someone, even indirectly, is personal data. A blockchain public key can fit this definition — and this is one of the most important debates in the field.

Propagation — "Spreading the word" After being signed, the transaction is sent to the network. Imagine you handed your signed form not to a notary's office, but to a public square full of people. Each person who receives the information passes it on to others.

In a few seconds, thousands of computers (called "nodes" of the network) already have a copy of your transaction. It is not yet confirmed — it is in a "waiting room" called a mempool, waiting to be processed.

It's as if the notary's office had a queue: your document has been received, but it hasn't been stamped yet.

Validation — "Checking if everything is correct" This is where the notary's office replaces the notary. Specialized computers (in the case of Bitcoin, called miners; in other networks, validators) take transactions from the waiting room and check:

Is the digital signature valid? Does the sender actually have what they claim to have?

Does the transaction follow all the network rules?

If something is wrong—a forged signature, insufficient funds, any irregularity—the transaction is rejected. No appeal, no "workaround." The rules are the same for everyone.

Imagine a group of independent accountants, each checking the same spreadsheet. If the majority agree that the accounts balance, the transaction is approved.

Compliance connection: this decentralized validation is fascinating from the perspective of Article 46 of the LGPD (Brazilian General Data Protection Law), which requires "security measures capable of protecting personal data." Distributed consensus validation is, technically, one of the most robust ways to guarantee data integrity that we have ever invented. But who is the "controller" responsible for this validation? The entire network? Nobody? That's another open question.

Inclusion in the Block — "Stamped and Archived Forever" Approved transactions are grouped into a block — think of it as a page in a notary's ledger. This page receives a unique stamp (called a hash) that functions like a fingerprint: if someone tries to alter even a comma on the page, the stamp changes completely, and everyone notices the fraud.

But the crucial detail is this: each new page contains the stamp of the previous page. This creates a chain — hence the name blockchain (chain of blocks). To alter an old transaction, you would need to redo all subsequent pages on thousands of computers simultaneously. In practice, this is unfeasible.

The transaction is now permanent. It's in the ledger. The ledger is in thousands of places. No one can tear the page out.

Connection with compliance: and this is exactly where the paradox lies. Article 18, item VI of the LGPD (Brazilian General Data Protection Law) guarantees the data subject the right to the deletion of personal data. But how do you delete something from a book that, by design, was made to be unalterable? This conflict between technical immutability and the legal right to be forgotten is the theme we will explore throughout this campaign.

What I learned building DPO2U
When I started developing DPO2U — a tool that attempts to reconcile blockchain with data protection — I thought the biggest challenge would be technical. Code, contracts, infrastructure.

I was wrong. The biggest challenge is conceptual. It's understanding that blockchain wasn't designed with privacy in mind. It was designed with transparency and immutability in mind. And the LGPD (Brazilian General Data Protection Law) wasn't designed with blockchain in mind. It was designed with traditional databases in mind that can be edited and deleted.

These two logics are on a collision course. And understanding the mechanics of a transaction — how it is created, validated, and becomes permanent — is the first step in understanding why this collision is so difficult to resolve.

Where are we going tomorrow?
Now that you know how a transaction becomes irreversible, the natural question is: what if I need to delete this data? If the LGPD (Brazilian General Data Protection Law) gives me the right to request the deletion of my information, how does this work in a system that was built to never forget anything?

On Day 3, we'll dive headfirst into this conflict: the immutability of the blockchain versus the right to be forgotten. This is where the privacy paradox gets really interesting.

What is Blockchain: A Technical Guide for Compliance Professionals

Fred Santana — Mon, 23 Mar 2026 14:11:27 +0000

Why a DPO Needs to Understand Blockchain

I’m Fred, founder of DPO2U. In recent months, I’ve been publicly building an on-chain compliance ecosystem, and I’ve encountered a problem that few admit exists: the technology that promises perfect auditability can violate the very law it’s supposed to protect.

This conflict has a name in academic literature. De Filippi and Wright, in Blockchain and the Law (2018), already warned that immutability would create direct tensions with the right to data deletion. In my Zettelkasten, I registered this as the Paradox of Immutability (ZK-20260125-013): the virtue of blockchain for integrity is its vice when personal data needs to be deleted.

This article is Day 1 of 30 in the “The Privacy Paradox” campaign. My goal: to explain, without unnecessary jargon, what you need to know about blockchain to make informed decisions. Not to become a programmer, but to avoid being deceived by those who sell magical solutions.

The Promise: Why Blockchain Matters for Compliance

Imagine your company needs to prove it obtained consent from a data subject on a specific date. Today, this proof is usually in an internal database controlled by the company itself. If there is litigation, the other party might question: “who guarantees that this record hasn’t been altered?”

In the DPO2U Whitepaper, I documented the vulnerabilities of the current system: privacy policies in editable PDFs without a trace, consent records in deletable spreadsheets, DPIAs with manual versioning. Which version is valid?

With blockchain, the consent record would be in an immutable ledger verifiable by third parties. No one would need to trust the company’s word. Think about the applications:

Consent record with an inviolable timestamp, an audit trail that no system administrator can tamper with, proof of compliance verifiable by regulators without depending on the company’s good faith.

It’s like replacing that Excel file that “proves” compliance with a digital notary record that no one controls alone.

Public vs. Private Blockchain: The Distinction That Matters

Not all blockchains work the same way, and this distinction is crucial for compliance.

A public blockchain functions like a mural in a public square. Anyone can read all the records. Bitcoin and Ethereum are examples. In them, recording a CPF (Brazilian tax identification number) or email address would be like writing personal data on a billboard, visible forever to everyone.

A private (or permissioned) blockchain functions more like the internal system of a consortium of companies. Only authorized participants can read and write. There is more control, but less decentralization.

For compliance professionals, the key question facing any blockchain solution is: who can read what is recorded? If the answer is “anyone on the internet,” Article 46 of the LGPD (Brazilian General Data Protection Law), which requires technical measures to protect personal data, is at risk.

Five Questions Every DPO Should Ask Before Accepting a Blockchain Solution

Is personal data on-chain or off-chain? If the answer is on-chain, immediately question compliance with Articles 16 and 18 of the LGPD (Brazilian General Data Protection Law).
Is the blockchain public or permissioned? This defines who has access to the records and directly impacts Article 46 (security measures).
Is there a deletion or anonymization mechanism? Immutable blockchain + personal data = regulatory risk. Ask how the provider intends to handle deletion requests.
Who is the data controller? In decentralized networks, the chain of responsibility foreseen in Article 5 of the LGPD becomes unclear. Someone needs to be responsible for compliance.
What is the legal basis for processing? Article 7 of the LGPD requires a legal basis for any processing. “It’s on the blockchain” is not a legal basis.

Day 1 of 30 — “The Privacy Paradox” Campaign. References: De Filippi, P., & Wright, A. (2018). Blockchain and the Law. Harvard University Press. | LGPD, Law 13.709/2018. Next: Immutability vs. Right to be Forgotten