Managing Private State in Midnight: Patterns, Pitfalls, and Best Practices

One of the most common points of confusion for developers new to Midnight is the distinction between private and public state. On EVM chains, the distinction is simple: everything in contract storage is public, and anything you want "private" requires off-chain infrastructure (encrypted APIs, TEEs, etc.). On Midnight, privacy is a first-class concept at the protocol level — but it comes with a significantly different mental model for state management.

Getting this wrong isn't just a privacy leak — it can mean lost funds, broken contracts, or proofs that fail to generate. This article covers what private state actually is in Midnight, the commitment model that underpins it, common patterns for working with private data, and the pitfalls that will bite you if you don't understand the model deeply.

---

What Private State Actually Is

In Midnight's architecture, state is split between two domains:

Ledger state (public): Stored on-chain, visible to all nodes, included in the block's state root. This includes contract addresses, Merkle tree roots, nullifier sets, token balances (when using public tokens), and any other data explicitly declared in the ledger block of your Compact contract.

Private state (client-side): Stored only by the user who owns it. Never posted to the chain. The chain only ever sees a commitment (a cryptographic hash) to this data. The actual values — balances, ownership flags, counters — live in the user's local state database.

This is architecturally closer to Zcash's shielded notes than to anything in Ethereum. The chain doesn't know what your private balance is. It knows a commitment exists (some leaf in a Merkle tree) and hasn't been spent (its nullifier isn't in the spent set). The actual value bound to that commitment is known only to you.

// Public ledger state — visible to all
ledger {
  commitment_tree: MerkleTree<Field, 20>,  // commitments to private values
  nullifiers: Set<Field>,                   // spent commitments
  root_history: Vec<Field>,                // historical roots for async proofs
}

// Private state — client-side only, never on-chain
// Conceptually, this is what the user's wallet stores:
type PrivateNote = {
  value: Uint<64>,
  asset_id: Field,
  secret: Field,
  owner_pubkey: PublicKey,
  tree_position: Uint<32>,
  merkle_path: MerklePath<20>,
}

The critical implication: if the client loses their private state, they cannot recover it from the chain. The chain contains no information that would let anyone reconstruct which commitment corresponds to which value. Private state loss is permanent.

---

The Commitment Model

A commitment is a collision-resistant hash of a private value. The commitment is posted on-chain; the preimage stays local.

The standard commitment scheme in Midnight:

// Pedersen commitment or Poseidon hash (circuit-friendly)
fn commit(value: Uint<64>, secret: Field, owner: PublicKey) -> Field {
  poseidon_hash([value as Field, secret, pubkey_to_field(owner)])
}

Why hash the owner into the commitment? Without binding the owner, anyone who discovers your (value, secret) pair could create a valid withdrawal proof. Binding to the owner's public key means only someone with the corresponding private key can construct a valid proof.

The secret (also called the "blinding factor" or "randomness") is critical. Without it, commitments to common values (e.g., exactly 100 tokens) would be identical across users, allowing correlation. The secret must be:

• Cryptographically random (256-bit entropy)
• Unique per commitment (reusing secrets enables linkability)
• Stored durably (loss = fund loss)

Commitment Lifecycle

1. User computes commitment = hash(value, secret, pubkey)
2. User stores (value, secret, merkle_pos, path) locally
3. Contract receives commitment, inserts into tree
4. Chain state: tree grows by one leaf
5. [Later] User wants to spend: loads local note, generates proof
6. Proof verifies note exists in tree and isn't nullified
7. Nullifier = hash(secret, commitment) is published on-chain
8. Chain state: nullifier set grows
9. Local state: note marked as spent

---

Common Private State Patterns

Pattern 1: Private Balance

The simplest case — a single private token balance. Instead of one counter that increments/decrements, you maintain a set of UTXO-style notes.

// Each "private balance unit" is a note
type PrivateBalanceNote = {
  amount: Uint<64>,
  token: TokenId,
  secret: Field,
  owner: PublicKey,
}

// To send 50 tokens to Bob when you have one note of 100:
// 1. Create output note for Bob: hash(50, bob_secret, bob_pubkey)
// 2. Create change note for self: hash(50, self_new_secret, self_pubkey)
// 3. Nullify input note
// 4. Insert both output commitments

circuit private_transfer(
  witness input_amount: Uint<64>,
  witness input_secret: Field,
  witness owner_privkey: PrivateKey,
  witness merkle_path: MerklePath<20>,
  witness send_amount: Uint<64>,
  witness change_secret: Field,
  witness recipient_pubkey: PublicKey,
  witness recipient_secret: Field,   // recipient generates and shares this

  public input_nullifier: Field,
  public send_commitment: Field,
  public change_commitment: Field
) {
  let owner_pubkey = derive_pubkey(owner_privkey);
  let input_commitment = commit(input_amount, input_secret, owner_pubkey);

  // Verify input note exists in tree
  assert merkle_verify(ledger.commitment_tree.root(), input_commitment, merkle_path);

  // Verify nullifier
  assert hash(input_secret, input_commitment) == input_nullifier;

  // No inflation: input >= send + change
  assert send_amount + (input_amount - send_amount) == input_amount;
  let change_amount = input_amount - send_amount;

  // Verify output commitments
  assert commit(send_amount, recipient_secret, recipient_pubkey) == send_commitment;
  assert commit(change_amount, change_secret, owner_pubkey) == change_commitment;
}

Key detail: the recipient_secret is chosen by the recipient (or shared via encrypted memo) — the sender cannot spend the recipient's note because they don't have the recipient's private key.

Pattern 2: Private Ownership Flag

Boolean private state: "does this address own this NFT?" expressed as a private flag.

type OwnershipNote = {
  asset_id: Field,
  owner: PublicKey,
  secret: Field,
}

circuit prove_ownership(
  witness asset_id: Field,
  witness owner_privkey: PrivateKey,
  witness secret: Field,
  witness merkle_path: MerklePath<20>,

  public ownership_nullifier: Field,   // for single-use proofs
  public asset_id_public: Field        // reveal which asset
) {
  let owner_pubkey = derive_pubkey(owner_privkey);
  let commitment = hash(asset_id, owner_pubkey, secret);

  assert merkle_verify(ledger.commitment_tree.root(), commitment, merkle_path);
  assert asset_id == asset_id_public;

  // Nullifier for this specific proof instance (prevents reuse of same proof)
  assert hash(secret, commitment, asset_id) == ownership_nullifier;
}

This circuit proves ownership of asset_id_public without revealing the owner's identity.

Pattern 3: Private Counter

A monotonically increasing counter — e.g., tracking how many times something has been done.

The challenge: you can't just "increment" a commitment. Each state transition requires creating a new commitment that commits to the new value and nullifying the old one.

circuit increment_counter(
  witness current_count: Uint<32>,
  witness secret: Field,
  witness owner_privkey: PrivateKey,
  witness merkle_path: MerklePath<20>,
  witness new_secret: Field,

  public old_nullifier: Field,
  public new_commitment: Field,
  public action_type: Field   // what was counted (public for auditability)
) {
  let owner_pubkey = derive_pubkey(owner_privkey);
  let old_commitment = hash(current_count as Field, secret, pubkey_to_field(owner_pubkey));

  assert merkle_verify(ledger.commitment_tree.root(), old_commitment, merkle_path);
  assert hash(secret, old_commitment) == old_nullifier;

  let new_count = current_count + 1;
  assert hash(new_count as Field, new_secret, pubkey_to_field(owner_pubkey)) == new_commitment;

  // Overflow protection
  assert new_count > current_count;
}

Each increment is a state transition: nullify old commitment, insert new one with count+1.

---

State Transitions That Preserve Privacy

Privacy-preserving state transitions must satisfy several properties:

1. No linkability between input and output commitments. An observer should not be able to tell that nullifier X corresponds to commitment Y. This is guaranteed by the ZK proof — the link is proven in zero-knowledge, so only the prover knows it.

2. Fresh secrets per commitment. Reusing a secret s across two commitments hash(v1, s, pk) and hash(v2, s, pk) means anyone who learns s can link both notes. Always generate fresh randomness.

3. Atomic transitions. In a transfer, the input nullification and output commitment insertion must happen in the same transaction. If they're split, there's a window where the input is spent but the output doesn't exist yet (fund loss), or the input isn't spent and the output exists (inflation).

contract PrivateState {
  fn transition(
    proof: ZKProof,
    old_nullifier: Field,
    new_commitments: Vec<Field>  // up to 2: recipient + change
  ) {
    // Atomicity: verify proof first, then apply state changes
    verify_proof(proof, transition_circuit, 
      [old_nullifier] + new_commitments);

    // Check nullifier is fresh
    assert !ledger.nullifiers.contains(old_nullifier);

    // Apply state changes atomically
    ledger.nullifiers.insert(old_nullifier);
    for commitment in new_commitments {
      ledger.commitment_tree.insert(commitment);
    }
    // Either all changes happen or none do (transaction atomicity)
  }
}

---

What Happens When Private State Is Lost

If a user loses their local private state (their note database), they cannot:

• Prove ownership of their commitments
• Generate withdrawal proofs
• Transfer their assets

The chain only stores commitments — opaque blobs with no decryptable content. Without the preimage (value, secret, owner), the commitment is unspendable.

Recovery Patterns

1. Encrypted backup with viewing key

The most robust approach: when creating a commitment, also produce an encrypted "memo" containing the note data, encrypted to a viewing key you control.

// In the deposit circuit, also produce an encrypted note
circuit deposit_with_memo(
  witness amount: Uint<64>,
  witness secret: Field,
  witness owner_pubkey: PublicKey,
  witness viewing_key: PublicKey,  // for recovery

  public commitment: Field,
  public encrypted_note: EncryptedMemo  // stored in calldata or event
) {
  assert commit(amount, secret, owner_pubkey) == commitment;

  // Encrypt note data to viewing key
  // The encrypted memo is posted publicly but readable only by viewing key holder
  let note_plaintext = encode(amount, secret, owner_pubkey);
  assert encrypt(note_plaintext, viewing_key) == encrypted_note;
}

The encrypted memo is stored in transaction calldata (not ledger state — it's cheap). Recovery: scan the chain for encrypted memos, decrypt with viewing key, reconstruct note database.

2. Deterministic secret derivation

If secrets are derived deterministically from a root seed:

secret_for_deposit_n = HMAC(root_seed, "deposit_" || n)

Recovery is possible by re-scanning the chain, recomputing secrets for all possible n values, checking if commit(?, secret_n, owner_pubkey) matches any commitment in the tree.

This requires scanning all commitments and trying all possible amounts — feasible for bounded amounts (e.g., integer USDC amounts 1-1,000,000) but not for arbitrary amounts.

3. Social recovery / guardian scheme

Split the viewing key among trusted parties using Shamir's secret sharing. Recovery requires k-of-n guardians to cooperate. More complex but eliminates single point of failure.

---

Pitfalls

1. State Bloat

Every deposit creates a new leaf in the commitment tree. Trees grow monotonically — you can't remove spent leaves without breaking the root history. In a long-running application with thousands of users making thousands of transactions, the tree can grow to millions of leaves.

The impact: Merkle path generation becomes expensive. At depth 20 (1M leaves), generating a path requires computing ~20 hashes. At depth 30 (1B leaves), it's 30 hashes but the tree itself requires 2GB of storage for a full node.

Mitigation: cap tree depth, archive old roots, or use sparse Merkle trees where the depth is fixed but empty subtrees are efficiently represented.

2. Proof Generation Time

ZK circuits with deep Merkle trees and complex arithmetic take measurable time to prove:

• Depth 16 tree, simple transfer: ~1-2 seconds on M2 MacBook
• Depth 20 tree, complex swap: ~5-10 seconds
• Depth 24 tree, aggregated proof: 30+ seconds

For interactive applications, this latency is noticeable. Consider:

• Pre-generating proofs for predictable operations
• Delegating proof generation to a trusted server (if you accept the trust assumption)
• Using simpler circuits where possible

3. Correlation Attacks

Even with perfect ZK privacy, metadata can leak:

• Timing correlation: If Alice deposits at time T and Bob withdraws at time T+5min, observers may suspect a connection
• Amount correlation: If you deposit 100 tokens and withdraw exactly 100 tokens, observers may link the transactions despite no cryptographic linkage
• Address reuse: Depositing from address A and receiving withdrawals at address A links all your transactions

Mitigations: introduce time delays, use round amounts or break/merge notes, use fresh addresses for each transaction.

4. Stale Merkle Roots

Merkle inclusion proofs are valid only for the root at the time of proof generation. If the tree updates between proof generation and proof submission (because another user makes a transaction), your proof is invalid — it references an old root.

Solution: maintain a root history on-chain. The contract accepts proofs against any root that was valid in the last N blocks.

ledger {
  commitment_tree: MerkleTree<Field, 20>,
  root_history: CircularBuffer<Field, 100>,  // keep last 100 roots
}

fn verify_inclusion(commitment: Field, path: MerklePath<20>, claimed_root: Field) {
  // Accept proofs against any recent root
  assert ledger.root_history.contains(claimed_root);
  assert merkle_verify(claimed_root, commitment, path);
}

This adds 100 root hashes of storage (3,200 bytes) and allows proof generation up to 100 blocks before submission — plenty for typical network conditions.

---

Summary

Private state in Midnight is client-side data bound to on-chain commitments. The commitment model — post a hash, prove knowledge of the preimage when spending, publish nullifiers to prevent double-spends — is the foundational pattern for everything from token balances to ownership flags to counters. The key discipline: manage local state carefully (it's not recoverable from the chain), use fresh secrets per commitment, and design state transitions atomically. The pitfalls — state bloat, proof latency, correlation attacks, stale roots — all have known mitigations, but they require deliberate design choices rather than afterthoughts.