DEV Community

Wilson
Wilson

Posted on • Originally published at gist.github.com

Understanding Wallet Sync: Why Your Midnight Deploy Fails Before It Starts

#blockchain #web3 #tutorial #midnight

Understanding Wallet Sync: Why Your Deploy Fails Before It Starts

You just wrote your first Midnight dApp. The code compiles, the proof server is running, you call balanceUnboundTransaction — and nothing happens. Or worse, you get a cryptic error about missing UTXOs. The problem isn't your contract. The problem is that your wallet hasn't synced.

This tutorial explains what wallet sync actually means on Midnight, why skipping it breaks everything, and the exact patterns to ensure your transactions always work.

What Is Wallet Sync?

Midnight's wallet doesn't maintain a live connection to the chain state. Instead, it synchronizes periodically by fetching the latest block data from the indexer. During sync, the wallet:

  1. Downloads new blocks from the indexer since the last sync point
  2. Scans for relevant UTXOs — shielded outputs encrypted to your keys, unshielded tokens in your address, and DUST fee tokens
  3. Updates local state — the wallet's view of which UTXOs it owns and can spend

When sync is complete, state.isSynced === true. When it's not, the wallet's view of the chain is stale. It may not know about UTXOs it just received, or it may try to spend UTXOs that were already consumed.

The Three Sub-Wallets

A Midnight wallet manages three independent sub-wallets, each with its own set of UTXOs and sync requirements:

1. Shielded Sub-Wallet

The shielded wallet holds private tokens. These are encrypted UTXOs that only the wallet owner can decrypt and spend. When someone sends you shielded tokens, the wallet must scan new blocks to discover and decrypt the relevant UTXOs.

Shielded tokens: Private, encrypted UTXOs
Discovery: Requires scanning blocks for encrypted outputs matching your keys
Sync dependency: HIGH — cannot spend what hasn't been decrypted
Enter fullscreen mode Exit fullscreen mode

2. Unshielded Sub-Wallet

The unshielded wallet holds transparent tokens — similar to a regular blockchain address. While these are visible on-chain, the wallet still needs to sync to know the current set of spendable UTXOs.

Unshielded tokens: Transparent, visible on-chain
Discovery: Requires indexing to find UTXOs at your address
Sync dependency: MEDIUM — chain data is public but wallet must still index it
Enter fullscreen mode Exit fullscreen mode

3. DUST Sub-Wallet

DUST is Midnight's fee token. Every transaction costs DUST, and the DUST sub-wallet must have available UTXOs to pay those fees. A wallet generates DUST by holding NIGHT tokens, but the wallet must sync to discover newly generated DUST UTXOs.

DUST tokens: Fee payment capacity, generated by holding NIGHT
Discovery: Requires scanning for DUST UTXOs generated at your address
Sync dependency: CRITICAL — no DUST sync means no fee payment means no transactions
Enter fullscreen mode Exit fullscreen mode

Why All Three Matter

When you call balanceUnboundTransaction, the wallet needs current UTXOs from all applicable sub-wallets to balance the transaction. If any sub-wallet is out of sync:

  • Unsynced shielded: The wallet may not know it received tokens, causing "insufficient balance" errors for funds that actually exist
  • Unsynced unshielded: Same problem for transparent tokens
  • Unsynced DUST: The wallet cannot pay transaction fees, and the transaction fails with a fee error

What Happens When You Skip Sync

Here's the failure cascade when you call balanceUnboundTransaction before sync completes:

Scenario 1: Missing UTXOs 

Time 0: Wallet created, sync starts
Time 1: You call balanceUnboundTransaction (sync NOT complete)
Result: Wallet uses stale UTXO set → "insufficient balance" error
Enter fullscreen mode Exit fullscreen mode

The wallet doesn't know about the UTXOs it recently received. It tries to build a transaction with an incomplete set of inputs, and the balancing fails because the inputs don't add up to the required outputs plus fees.

Scenario 2: Double-Spending Stale UTXOs 

Time 0: Transaction A submitted, consumes UTXO #1
Time 1: Wallet hasn't re-synced, still thinks UTXO #1 is available
Time 2: You call balanceUnboundTransaction for Transaction B
Result: Wallet tries to spend UTXO #1 again → error 1010 (invalid transaction)
Enter fullscreen mode Exit fullscreen mode

If a previous transaction consumed a UTXO but the wallet hasn't synced to learn about that consumption, it may try to use the same UTXO again. The network rejects this as a double-spend.

Scenario 3: DUST Fee Failure 

Time 0: Wallet receives NIGHT tokens, which begin generating DUST
Time 1: You call transact() (DUST sub-wallet not yet synced)
Result: Wallet has no known DUST UTXOs → cannot pay fees → transaction fails
Enter fullscreen mode Exit fullscreen mode

New wallets are especially vulnerable. Until the wallet syncs and discovers its DUST UTXOs, it literally cannot pay for any transaction.

The Known DUST Wallet Bug

There is a known issue with the DUST sub-wallet's sync on idle chains. When the network has low activity, the DUST wallet's isStrictlyComplete() method may hang indefinitely, never resolving to true.

What Happens

The wallet sync process checks whether each sub-wallet has finished processing all relevant blocks. For shielded and unshielded wallets, this check works reliably. But for the DUST sub-wallet on a quiet network:

// This may NEVER resolve on an idle chain:
const state = await wallet.state();
// state.isSynced may stay false indefinitely
// because isStrictlyComplete() hangs waiting for
// DUST-related block data that arrives slowly
Enter fullscreen mode Exit fullscreen mode

The root cause is that isStrictlyComplete() waits for confirmation that all expected DUST-generating events have been processed. On an idle chain, these events are sparse, and the completion signal may not arrive.

Workaround: Use isSynced with Timeout

Instead of relying on isStrictlyComplete(), use the isSynced flag with a timeout:

import { filter, take, timeout } from '@midnight-ntwrk/rx';

async function waitForWalletSync(
  wallet: Wallet & Resource,
  timeoutMs: number = 60_000
): Promise<WalletState> {
  try {
    const syncedState = await wallet.state()
      .pipe(
        filter((state: WalletState) => state.isSynced),
        take(1),
        timeout(timeoutMs)
      )
      .toPromise();
    return syncedState;
  } catch (err) {
    // Timeout — wallet may still be partially synced
    // Check current state as a fallback
    const currentState = await wallet.state().pipe(take(1)).toPromise();
    if (currentState.isSynced) {
      return currentState;
    }
    throw new Error(
      `Wallet sync timed out after ${timeoutMs}ms. ` +
      `Current state: synced=${currentState.isSynced}`
    );
  }
}
Enter fullscreen mode Exit fullscreen mode

This pattern:

  1. Waits for isSynced === true with a timeout
  2. Falls back to checking current state if timeout occurs
  3. Throws a clear error if the wallet genuinely can't sync

Alternative: Polling Check

For environments where RxJS observables are awkward, a polling approach works:

async function waitForSyncByPolling(
  wallet: Wallet & Resource,
  maxAttempts: number = 30,
  intervalMs: number = 2000
): Promise<void> {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    const state = await new Promise<WalletState>((resolve) => {
      const sub = wallet.state().subscribe({
        next: (s) => {
          sub.unsubscribe();
          resolve(s);
        },
      });
    });

    if (state.isSynced) {
      console.log(`[Sync] Wallet synced after ${attempt + 1} attempts`);
      return;
    }

    await new Promise((resolve) => setTimeout(resolve, intervalMs));
  }

  throw new Error(`Wallet failed to sync after ${maxAttempts} attempts`);
}
Enter fullscreen mode Exit fullscreen mode

The Safe Sync Pattern

Every production Midnight application should follow this pattern:

1. Wait for Sync at Startup 

async function initializeApplication(config: AppConfig): Promise<AppContext> {
  const wallet = await createWallet(config.seed, {
    networkId: config.networkId,
    indexer: config.indexer,
    indexerWs: config.indexerWs,
    proofServer: config.proofServer,
    rpc: config.rpc,
  });

  // ALWAYS wait for sync before accepting requests
  console.log('[Startup] Waiting for wallet sync...');
  const syncedState = await waitForWalletSync(wallet, 60_000);
  console.log('[Startup] Wallet synced. Address:', wallet.ownPublicKey().toString());

  const providers = createProviders(wallet);

  return { wallet, providers, syncedState };
}
Enter fullscreen mode Exit fullscreen mode

2. Check Sync Before Every Transaction 

async function submitWithSyncCheck(
  wallet: Wallet & Resource,
  operation: () => Promise<Transaction>
): Promise<SubmitResult> {
  // Verify wallet is synced before starting
  const state = await wallet.state().pipe(take(1)).toPromise();

  if (!state.isSynced) {
    console.warn('[Sync] Wallet not synced, waiting...');
    await waitForWalletSync(wallet, 30_000);
  }

  // Now safe to transact
  return submitContractTransaction(wallet, providers, operation);
}
Enter fullscreen mode Exit fullscreen mode

3. Re-Sync After Failures

When a transaction fails with a UTXO-related error, re-sync before retrying:

async function transactWithRetry(
  wallet: Wallet & Resource,
  operation: () => Promise<Transaction>,
  maxRetries: number = 3
): Promise<SubmitResult> {
  let lastError: Error | null = null;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      // Re-sync on retries (not on first attempt, we already synced)
      if (attempt > 0) {
        console.log(`[Retry ${attempt}] Re-syncing wallet...`);
        await waitForWalletSync(wallet, 30_000);
      }

      return await submitWithSyncCheck(wallet, operation);
    } catch (error) {
      lastError = error instanceof Error ? error : new Error(String(error));

      // Don't retry logic errors
      if (isLogicError(lastError)) {
        throw lastError;
      }

      console.warn(`[Retry ${attempt}] Transaction failed: ${lastError.message}`);
    }
  }

  throw lastError;
}

function isLogicError(error: Error): boolean {
  return (
    error.message.includes('insufficient') ||
    error.message.includes('circuit') ||
    error.message.includes('proof')
  );
}
Enter fullscreen mode Exit fullscreen mode

Sync in the Prove-Balance-Submit Pipeline

The complete transaction pipeline with sync checks looks like this:

┌──────────────┐     ┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│  0. SYNC     │────→│  1. TRANSACT │────→│  2. PROVE    │────→│  3. BALANCE  │
│  wallet.state│     │  create tx   │     │  proveTx    │     │  balance tx  │
│  isSynced?   │     │              │     │              │     │              │
└──────────────┘     └──────────────┘     └──────────────┘     └──────────────┘
                                                                     │
                                                             ┌───────▼───────┐
                                                             │  4. SUBMIT    │
                                                             │  submitTx     │
                                                             │               │
                                                             └───────────────┘
Enter fullscreen mode Exit fullscreen mode

Step 0 is the sync check. Without it, steps 1–4 operate on stale data.

Practical Monitoring

For production backends, monitor wallet sync health continuously:

import { filter, take } from '@midnight-ntwrk/rx';

class SyncMonitor {
  private lastSyncTime: Date | null = null;
  private isSynced: boolean = false;

  start(wallet: Wallet & Resource, alertCallback: (msg: string) => void): void {
    wallet.state().subscribe({
      next: (state) => {
        const wasSynced = this.isSynced;
        this.isSynced = state.isSynced;

        if (state.isSynced && !wasSynced) {
          this.lastSyncTime = new Date();
          console.log('[SyncMonitor] Wallet synced at', this.lastSyncTime.toISOString());
        } else if (!state.isSynced && wasSynced) {
          console.warn('[SyncMonitor] Wallet desynced');
          alertCallback('Wallet lost sync — transactions may fail');
        }
      },
      error: (err) => {
        console.error('[SyncMonitor] State subscription error:', err);
        alertCallback(`Wallet state error: ${err.message}`);
      },
    });
  }

  getStatus(): { isSynced: boolean; lastSyncTime: Date | null } {
    return {
      isSynced: this.isSynced,
      lastSyncTime: this.lastSyncTime,
    };
  }
}
Enter fullscreen mode Exit fullscreen mode

Integrate this into your health check endpoint:

app.get('/health', (_req, res) => {
  const syncStatus = syncMonitor.getStatus();
  res.json({
    status: 'ok',
    walletSynced: syncStatus.isSynced,
    lastSyncTime: syncStatus.lastSyncTime?.toISOString() ?? null,
  });
});
Enter fullscreen mode Exit fullscreen mode

Common Pitfalls

Pitfall 1: Ignoring Sync on Startup 

// WRONG: Immediately accepting requests
const wallet = await createWallet(seed, config);
app.listen(3000); // Wallet may not be synced yet!

// RIGHT: Wait for sync before serving traffic
const wallet = await createWallet(seed, config);
await waitForWalletSync(wallet, 60_000);
app.listen(3000);
Enter fullscreen mode Exit fullscreen mode

Pitfall 2: Not Re-Syncing After Failures

When a transaction fails with "insufficient balance" or "invalid transaction", the wallet's UTXO set may be stale. Re-sync before retrying:

// WRONG: Retry immediately with stale state
for (let i = 0; i < 3; i++) {
  try { return await transact(); } catch { continue; }
}

// RIGHT: Re-sync between retries
for (let i = 0; i < 3; i++) {
  try {
    return await transact();
  } catch (error) {
    if (i < 2) {
      await waitForWalletSync(wallet, 30_000); // Re-sync
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Pitfall 3: Assuming isSynced Stays True

Sync is not permanent. The wallet can fall out of sync if:

  • The indexer connection drops
  • The network has a reorganization
  • The wallet process was paused or slept

Monitor the sync state continuously, not just at startup.

Pitfall 4: Using the Wrong Sync Method 

// WRONG: This returns the CURRENT state snapshot, not a promise that resolves
// when sync completes. If the wallet isn't synced, this returns immediately
// with isSynced = false.
const state = await wallet.state().pipe(take(1)).toPromise();
if (!state.isSynced) {
  // Wallet is NOT synced. Do not transact.
}

// RIGHT: Wait for sync to complete
const syncedState = await wallet.state()
  .pipe(
    filter((s: WalletState) => s.isSynced),
    take(1)
  )
  .toPromise();
Enter fullscreen mode Exit fullscreen mode

Startup Health Check Integration

Combine wallet sync waiting with proof server health checks for a robust startup sequence:

async function startServer(): Promise<void> {
  // 1. Check proof server
  console.log('[Startup] Checking proof server...');
  const proofServerHealthy = await checkProofServerHealth();
  if (!proofServerHealthy) {
    throw new Error('Proof server is not reachable');
  }

  // 2. Initialize wallet
  console.log('[Startup] Initializing wallet...');
  const wallet = await createWallet(seed, walletConfig);

  // 3. Wait for sync (with DUST bug workaround)
  console.log('[Startup] Waiting for wallet sync...');
  const syncedState = await waitForWalletSync(wallet, 60_000);
  console.log('[Startup] Wallet synced:', {
    address: wallet.ownPublicKey().toString(),
    isSynced: syncedState.isSynced,
  });

  // 4. Start accepting requests
  app.listen(3000, () => {
    console.log('[Server] Ready on port 3000');
  });
}
Enter fullscreen mode Exit fullscreen mode

Summary

Pattern When to Use Key Code
Startup sync wait App initialization wallet.state().pipe(filter(s => s.isSynced), take(1)).toPromise()
Pre-transaction check Before every balanceUnboundTransaction Check state.isSynced first
Re-sync on failure After UTXO-related errors Re-call waitForWalletSync before retry
Continuous monitoring Production health checks wallet.state().subscribe(...)
Timeout workaround DUST wallet hang on idle chains Use timeout() RxJS operator

The rule is simple: never call balanceUnboundTransaction without confirming isSynced === true first. If you follow this pattern, you'll avoid the most common class of Midnight deployment failures.

Top comments (0)