DEV Community

Tosh
Tosh

Posted on

Handling Midnight SDK Breaking Changes: A Developer's Survival Guide

#web3 #blockchain #javascript #tutorial

Handling Midnight SDK Breaking Changes: A Developer's Survival Guide

Every developer building on early-stage blockchain SDKs has lived through this: you run npm install after a team member updates the lockfile, and suddenly nothing compiles. Midnight's SDK is actively evolving, and breaking changes are frequent enough that you need a systematic approach to handling them.

This guide covers practical strategies for managing Midnight SDK upgrades — from detecting breaking changes before they break production, to a step-by-step migration workflow, to maintaining backward compatibility when you can't upgrade immediately.

Why Midnight SDK Changes Break More Than Normal Libraries

Midnight SDK upgrades are more disruptive than typical library updates for a structural reason: proving keys are cryptographically bound to specific circuit versions.

When you upgrade the SDK, even a minor version bump can:

  1. Change the circuit constraint system
  2. Invalidate your existing proving keys
  3. Require regeneration of proving and verification keys
  4. Break existing proofs generated against the old keys

Unlike a typical API change where you update a function call, a circuit change means any proofs generated before the upgrade are incompatible with contracts deployed after it.

This creates a hard migration surface on two dimensions:

  • Your development environment: compilation and proof generation
  • Your deployment state: on-chain contracts vs. client-generated proofs

Step 1: Track Changes Before Upgrading

Never upgrade blindly. Before changing any @midnight-ntwrk/* version:

Check the changelog:

# In your repo, view the current versions
cat package.json | grep "@midnight-ntwrk"

# On npm, check what changed between versions
npx npm-check-updates --filter "@midnight-ntwrk/*" --dry-run
Enter fullscreen mode Exit fullscreen mode

Read the release notes directly:

The Midnight SDK publishes changelogs at: https://github.com/midnight-ntwrk/midnight-js/releases

Look specifically for entries labeled:

  • BREAKING CHANGE — requires code changes
  • Circuit change — requires key regeneration
  • API rename — function/interface names changed
  • Type change — TypeScript types modified (common source of silent breakage)

Check the migration guide:
If a migration guide exists, read it in full before starting. The 15 minutes of reading saves hours of debugging.

Step 2: Pin Exact Versions (Never Use ^ or ~) 

// DANGEROUS — floating versions
{
  "dependencies": {
    "@midnight-ntwrk/compact-runtime": "^0.14.0",
    "@midnight-ntwrk/midnight-js-types": "~0.14.0"
  }
}

// CORRECT — pinned exact versions
{
  "dependencies": {
    "@midnight-ntwrk/compact-runtime": "0.14.0",
    "@midnight-ntwrk/midnight-js-types": "0.14.0",
    "@midnight-ntwrk/midnight-js-contracts": "0.14.0",
    "@midnight-ntwrk/midnight-js-network-id": "0.14.0"
  }
}
Enter fullscreen mode Exit fullscreen mode

Lockfile discipline:
Commit package-lock.json (or yarn.lock). Never .gitignore it. Your CI should install with npm ci (not npm install) to use the exact lockfile versions.

Step 3: Upgrade in an Isolated Branch

Never upgrade the Midnight SDK directly on main. Create an upgrade branch:

git checkout -b upgrade/midnight-sdk-0.15.0
Enter fullscreen mode Exit fullscreen mode

This lets you:

  • Run the full migration without disrupting other work
  • Revert cleanly if something breaks badly
  • Get a code review of the upgrade diff before merging

Step 4: Update Versions and Recompile Contracts

Update package.json manually (don't use npm update with floating versions):

# Update to specific new version
npm install @midnight-ntwrk/compact-runtime@0.15.0 \
  @midnight-ntwrk/midnight-js-types@0.15.0 \
  @midnight-ntwrk/midnight-js-contracts@0.15.0 \
  --save-exact
Enter fullscreen mode Exit fullscreen mode

Then recompile all Compact contracts:

# Recompile all .compact files
find ./contracts -name "*.compact" | while read contract; do
  echo "Compiling: $contract"
  npx compactc "$contract" -o "$(dirname "$contract")/build"
done
Enter fullscreen mode Exit fullscreen mode

If compilation fails, the error messages from compactc are usually clear about what changed. Common patterns:

Error: 'disclose' function signature changed in 0.15.0
# Fix: Update all disclose() calls to new signature

Error: Type 'Uint64' is no longer assignable to 'Field'
# Fix: Use explicit conversion via toField()
Enter fullscreen mode Exit fullscreen mode

Step 5: Regenerate Proving Keys

After any compilation success, regenerate proving and verification keys. Even if the circuit didn't visibly change, don't trust old keys.

# Script to regenerate all keys
#!/bin/bash
set -euo pipefail

CONTRACTS_DIR="./contracts"
KEYS_DIR="./keys"

mkdir -p "$KEYS_DIR"

for contract_dir in "$CONTRACTS_DIR"/*/build; do
  contract_name=$(basename "$(dirname "$contract_dir")")

  echo "Generating keys for: $contract_name"

  npx compact-cli keygen \
    --circuit "$contract_dir/circuit.json" \
    --proving-key "$KEYS_DIR/${contract_name}.pk" \
    --verification-key "$KEYS_DIR/${contract_name}.vk"

  echo "✅ Keys generated for $contract_name"
done

echo "All keys regenerated. Commit the new keys."
Enter fullscreen mode Exit fullscreen mode

Important: Proving keys must be committed to your repository. They're large (often 50-200MB), so use Git LFS if needed:

git lfs track "*.pk"
git lfs track "*.vk"
echo "*.pk filter=lfs diff=lfs merge=lfs -text" >> .gitattributes
echo "*.vk filter=lfs diff=lfs merge=lfs -text" >> .gitattributes
git add .gitattributes
git lfs install
Enter fullscreen mode Exit fullscreen mode

Step 6: Update the Witness Implementation

After recompiling, review your TypeScript witness implementations. These are the most failure-prone area because TypeScript doesn't always catch circuit-level mismatches at compile time.

What to look for:

// Old pattern (pre-0.15.0)
import { buildTransferWitness } from '@midnight-ntwrk/compact-runtime';

// New pattern (post-0.15.0) — function renamed
import { createTransferWitness } from '@midnight-ntwrk/compact-runtime';
Enter fullscreen mode Exit fullscreen mode

Run the TypeScript compiler as a first check:

npx tsc --noEmit 2>&1 | head -50
Enter fullscreen mode Exit fullscreen mode

TypeScript errors reveal API changes. Fix all TypeScript errors before running tests.

Step 7: Run the Full Test Suite

After compilation and TypeScript checks pass, run your tests:

npm test
Enter fullscreen mode Exit fullscreen mode

Common test failure modes after upgrades:

  1. Constraint violations in tests: If tests fail with "constraint not satisfied," the witness implementation doesn't match the new circuit. Compare the old and new circuit constraint counts.

  2. Proof verification failures: Old proofs in test fixtures are invalid. Regenerate test fixtures.

  3. Timeout failures: A new proving system may be slower. Increase test timeouts:

   // In jest.config.ts
   export default {
     testTimeout: 120000, // 2 minutes for proof generation
   };
Enter fullscreen mode Exit fullscreen mode
  1. Network interaction failures: If the SDK changed how it connects to the network, update your test setup.

Step 8: Test on Local Devnet, Then Testnet

After local tests pass:

  1. Deploy to local devnet (if Midnight provides one)
  2. Run integration tests against the devnet
  3. Deploy to testnet
  4. Run smoke tests on testnet before committing to the upgrade

If you have existing testnet deployments, understand that you may need to redeploy contracts if the circuit changed. Old contracts with new client code will produce proof verification failures.

Handling "I Can't Upgrade Right Now" Situations

Sometimes you learn about a breaking SDK change but you can't migrate immediately (active user base, other ongoing work, etc.).

Strategy 1: Lock the environment completely

Prevent accidental upgrades at the system level:

# .npmrc — add to repo root
package-lock=true
engine-strict=true

# package.json engines field
"engines": {
  "node": ">=18.0.0 <20.0.0"
}
Enter fullscreen mode Exit fullscreen mode

Strategy 2: Document the freeze with a clear exit criteria

Create UPGRADE_BLOCKED.md in the repo:

# Midnight SDK Upgrade Blocked

Current version: 0.14.0
Target version: 0.15.0
Blocked since: 2026-01-15
Blocker: Active testnet deployment with 200+ users
Exit criteria: Testnet migration window scheduled for 2026-02-01
Owner: @yourname

## What breaks in 0.15.0
- [List specific breaking changes]
- [Associated code areas]

## Migration plan
[Brief migration plan when the window opens]
Enter fullscreen mode Exit fullscreen mode

Strategy 3: Run parallel environments

For longer freezes, maintain two deployment environments with different SDK versions. Route traffic based on contract version detected from on-chain state.

Diagnosing Proof Generation Failures After Upgrade

If proof generation fails after a successful upgrade, here's the diagnostic checklist:

1. Check proving key freshness

# Get the modification time of the compiled circuit
stat contracts/my-contract/build/circuit.json

# Compare with proving key
stat keys/my-contract.pk

# If circuit is newer than key: regenerate the key
Enter fullscreen mode Exit fullscreen mode

2. Verify circuit input types

# Inspect the circuit's public inputs
cat contracts/my-contract/build/circuit.json | python3 -c "
import json, sys
circuit = json.load(sys.stdin)
print('Public inputs:')
for name, type_info in circuit.get('public_inputs', {}).items():
    print(f'  {name}: {type_info}')
print('Private inputs:')
for name, type_info in circuit.get('private_inputs', {}).items():
    print(f'  {name}: {type_info}')
"
Enter fullscreen mode Exit fullscreen mode

Compare against your witness implementation and verify the types match exactly.

3. Test with minimal inputs

Isolate the failing constraint with a minimal test:

it('should prove minimal transfer', async () => {
  const proof = await proveTransfer({
    from: testKey,
    to: testAddress,
    amount: 1n,           // minimum amount
    balance: 1n,          // exactly equal to amount
  });
  expect(proof).toBeDefined();
});
Enter fullscreen mode Exit fullscreen mode

4. Enable verbose proving output

COMPACT_PROOF_VERBOSE=1 npm test 2>&1 | grep -E "constraint|witness|error"
Enter fullscreen mode Exit fullscreen mode

Automating Upgrade Detection

Set up a CI job to detect new SDK versions:

# .github/workflows/check-sdk-updates.yml
name: Check Midnight SDK Updates
on:
  schedule:
    - cron: '0 9 * * 1'  # Every Monday 9AM

jobs:
  check-updates:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - name: Check for SDK updates
        run: |
          npm install -g npm-check-updates
          UPDATES=$(ncu --filter "@midnight-ntwrk/*" --jsonUpgraded 2>/dev/null || echo "{}")
          if [ "$UPDATES" != "{}" ] && [ "$UPDATES" != "null" ]; then
            echo "New SDK versions available:"
            echo $UPDATES
            # Optionally create a GitHub issue or send a notification
          fi
Enter fullscreen mode Exit fullscreen mode

After going through this a few times, it gets less painful — you develop a feel for what's likely to break and where. The things that burned me most: skipping key regeneration because "I didn't change the circuit logic" (I did, indirectly), and not testing witness edge cases after an upgrade because the TypeScript compiled clean.

If you're building anything serious on Midnight right now, set up the weekly CI check for SDK updates and treat every version bump as a breaking change until proven otherwise. The proving key dependency is what makes these upgrades fundamentally different from normal library updates — once you've debugged a proof verification failure at 2am, you won't skip that step again.

Top comments (0)