Nasihudeen Jimoh for Midnight Aliit Fellowship

Posted on Apr 2

Advanced Compact Patterns for Web3 Developers

#architecture #blockchain #tutorial #web3

Introduction

If you've spent years building on EVM chains, Midnight's architecture might feel like a paradigm shift. On Ethereum, you push computation onto the blockchain itself. On Midnight, you do the opposite you move computation off-chain and prove it correctly using zero-knowledge proofs.

This isn't just a different implementation detail. It fundamentally changes how you think about state management, data disclosure, and circuit design.

Samantha's foundational guide introduced the three-part structure of Midnight contracts: the public ledger, zero-knowledge circuits, and local computation. But understanding the basics and architecting production systems are two different challenges.

This guide dives into the patterns that separate working prototypes from robust systems. We'll explore how witnesses enable privacy boundaries, why commitments matter more than direct state, how to optimize circuits for real-world constraints, and how to compose multiple private contracts without leaking metadata.

By the end, you'll have concrete strategies for building systems that maintain privacy guarantees while managing the practical tradeoffs of Web3 applications.

Witnesses & Selective Disclosure

Understanding Witnesses Beyond Function Calls

In EVM contracts, all data available to a function is deterministic. The blockchain is your single source of truth. In Compact, witnesses invert this model: witnesses are the only source of truth the contract doesn't control.

// A witness declares a contract's dependency on external data
witness getUserSecret(): Bytes<32>;
witness getProofOfAssets(userId: Uint<64>): AssetsProof;

When you declare a witness, you're saying: "This contract's logic depends on data I cannot verify on-chain. It's the application's responsibility to provide this correctly."

This creates a critical security boundary. The contract trusts the application to supply honest witnesses, but the proof system validates that the application used those witnesses correctly.

The Witness-Disclosure Loop

Real-world contracts don't just consume witnesses they combine witness data with disclosed state to create privacy preserving outcomes.

Consider an age verification system:

pragma language_version 0.22;
import CompactStandardLibrary;

// Public ledger: only record that someone proved they're eligible
export ledger ageVerified: Map<Bytes<32>, Boolean>;
export ledger verificationRound: Counter;

// Private witness: the user's actual birthdate (never on-chain)
witness getUserBirthDate(): Uint<32>; // Unix timestamp

// Derived public key with round counter to prevent replay
circuit derivePublicIdentity(round: Field, secret: Bytes<32>): Bytes<32> {
  return persistentHash<Vector<2, Bytes<32>>>(
    [round as Bytes<32>, secret]
  );
}

// Main circuit: prove age without revealing birthdate
export circuit verifyAge(secret: Bytes<32>, minAge: Uint<32>): [] {
  // Get private birthdate (witness - not on-chain)
  const birthDate = getUserBirthDate();

  // Compute age
  const currentTimestamp: Uint<32> = 1704067200; // Updated by app
  const age = currentTimestamp - birthDate;

  // Private check: verify age requirement
  assert(age >= minAge, "Age requirement not met");

  // Public disclosure: only record that verification happened
  const identity = derivePublicIdentity(verificationRound.roundNumber, secret);
  ageVerified = disclose(identity, true);
  verificationRound.increment(1);
}

Key pattern: The witness data (getUserBirthDate) is never directly disclosed. Instead, you compute a predicate over it (age >= minAge), and then disclose only the outcome the user consents to.

The tradeoff: The application code that supplies the witness must be trusted. If a malicious DApp sends a false birthdate, the proof system can't detect it—but it will prove the user accepted false data. This is why witness sourcing matters as much as circuit logic.

Practical Consideration: Witness Sourcing

Where do witnesses come from in real applications?

User-held secrets: API keys, private keys, personal data
External APIs: Proof-of-reserve attestations, oracle feeds, credential issuers
Zero-knowledge proofs themselves: A sub-proof generated off-chain that proves something about external data
Trusted hardware: TEE attestations or trusted execution environment outputs

Each source has different security properties. A witness from a user's secret key is as strong as that key's protection. A witness from an untrusted API might need cryptographic verification itself.

For example, if your witness comes from an API like "get current asset price," the application must either:

Trust the API (weak)
Verify the API response against multiple oracles (medium)
Require the API to provide a signature from a trusted source (better)
Use sub-proofs to prove the API data meets certain criteria without revealing it (best)

Commitments & Zero-Knowledge Proof Architecture

From State Mutation to Commitment Schemes

EVM developers are accustomed to direct state mutations:

// Ethereum: modify state directly
mapping(address => uint256) balance;
balance[user] += amount;

In Midnight, public state mutations must be proven by a zero-knowledge circuit. This means your public state must be designed around commitment schemes cryptographic structures that let you prove you know a value without revealing it.

Here's the conceptual bridge:

EVM thinking: State is a mutable cell. Update it directly.
Midnight thinking: State is a commitment to a value. Prove you know the value, then update the commitment.

Building a Private Ledger with Commitments

Let's walk through a private token transfer system:

pragma language_version 0.22;
import CompactStandardLibrary;

// Public state: only commitments to balances, never actual amounts
export ledger balanceCommitment: Map<Bytes<32>, Field>;
export ledger totalSupply: Uint<128>;
export ledger transferRound: Counter;

// Private data structure (never on-chain, only proven)
struct Account {
  owner: Bytes<32>,
  balance: Uint<128>,
  nonce: Uint<64>
}

// Witness: the actual account data, held privately by user
witness getAccount(): Account;
witness getNullifierSecret(): Bytes<32>;

// Helper: derive a nullifier to prevent double-spending
circuit deriveNullifier(nonce: Uint<64>, secret: Bytes<32>): Field {
  return persistentHash<Vector<2, Field>>(
    [persistentHash<Bytes<32>>(nonce as Bytes<32>), 
     persistentHash<Bytes<32>>(secret)]
  ) as Field;
}

// Helper: commitment to an account
circuit commitToAccount(account: Account, salt: Bytes<32>): Field {
  return persistentHash<Vector<2, Field>>(
    [persistentHash<Account>(account),
     persistentHash<Bytes<32>>(salt)]
  ) as Field;
}

// Main circuit: prove a valid token transfer
export circuit transfer(
  recipient: Bytes<32>,
  amount: Uint<128>,
  salt: Bytes<32>,
  newSalt: Bytes<32>
): [] {
  // Load private account data
  const account = getAccount();
  const nullifierSecret = getNullifierSecret();

  // Verify the account commitment exists
  const oldCommitment = commitToAccount(account, salt);
  assert(
    balanceCommitment[account.owner] == oldCommitment,
    "Account commitment mismatch"
  );

  // Private verification: user has sufficient balance
  assert(account.balance >= amount, "Insufficient balance");

  // Compute new account state (private)
  const newAccount: Account = [
    owner: account.owner,
    balance: account.balance - amount,
    nonce: account.nonce + 1
  ];

  // Create nullifier to prevent replay
  const nullifier = deriveNullifier(account.nonce, nullifierSecret);

  // Update public state with new commitment
  const newCommitment = commitToAccount(newAccount, newSalt);
  balanceCommitment = disclose(account.owner, newCommitment);

  // Record nullifier to prevent double-spend
  // In a real system, this would be a set of spent nullifiers
  // For now, we disclose it as proof of spending
  disclose(nullifier);

  // Recipient balance update (simplified: assume recipient pre-existed)
  // In production, you'd handle account creation
  transferRound.increment(1);
}

The Commitment Tradeoff

This approach provides strong privacy but requires careful design:

Advantages:

Balances are never visible on-chain (only commitments)
Transfers reveal no information except that a transfer occurred
The system is composable with other private circuits

Costs:

Every balance update requires a full commitment recomputation
Clients must store balance commitments locally (or query from a private oracle)
Replay protection requires tracking spent nullifiers
The circuit is more complex, leading to larger proofs and longer proving times

Real world consideration: For most applications, you won't implement full commitment schemes from scratch. You'll use Midnight's standard library, which provides optimized versions. But understanding the underlying structure helps you choose the right patterns for your use case.

Circuit Optimization

Why Circuit Optimization Matters

Compact circuits must be bounded at compile time. You can't have unbounded loops or recursive calls. This constraint exists because every circuit must compile to a fixed-size zero-knowledge proof.

For EVM developers, this is a significant mindset shift. On Ethereum, you pay gas for computation. On Midnight, you accept predetermined computation bounds.

// This won't compile unbounded recursion
circuit traverse(node: TreeNode): Uint<64> {
  if (node.left == null) {
    return node.value;
  } else {
    return traverse(node.left);
  }
}

// This works—bounded by tree depth
export circuit traverseFixed<#DEPTH>(
  node: TreeNode, 
  path: Vector<#DEPTH, Boolean>
): Uint<64> {
  let current = node;
  for (let i = 0; i < #DEPTH; i++) {
    if (path[i]) {
      current = current.right; // Assumes node structure allows this
    } else {
      current = current.left;
    }
  }
  return current.value;
}

Optimization Strategies

1: Vectorization and Batching

For operations on multiple items, vectorize instead of looping:

// Less efficient: separate proofs for each item
export circuit verifyAge(secret: Bytes<32>, minAge: Uint<32>): [] {
  const birthDate = getUserBirthDate();
  assert(currentTime - birthDate >= minAge, "Too young");
}

// More efficient: batch verification
export circuit verifyAgesInBatch<#N>(
  secrets: Vector<#N, Bytes<32>>,
  minAges: Vector<#N, Uint<32>>
): [] {
  for (let i = 0; i < #N; i++) {
    // Witness supplies ages for all users
    const birthDates = getUserBirthDates(i);
    assert(
      currentTime - birthDates >= minAges[i],
      "Age check failed"
    );
  }
}

2: Lazy Evaluation with Merkle Trees

Instead of processing all data inline, use Merkle trees to prove membership:

// Direct approach: verify all items (O(n) circuit size)
export circuit verifyAllBalances<#N>(
  balances: Vector<#N, Uint<128>>,
  totalRequired: Uint<128>
): [] {
  let sum: Uint<128> = 0;
  for (let i = 0; i < #N; i++) {
    sum = sum + balances[i];
  }
  assert(sum >= totalRequired, "Insufficient total");
}

// Optimized: verify membership in Merkle tree (O(log n) circuit size)
export circuit verifyBalanceProof(
  balance: Uint<128>,
  merkleProof: Vector<32, Field>, // Log2(2^32) = 32 levels
  merkleRoot: Field,
  leaf_index: Uint<32>
): [] {
  // Recompute leaf and verify path
  const leaf = persistentHash<Uint<128>>(balance) as Field;
  let current = leaf;

  for (let i = 0; i < 32; i++) {
    const proofElement = merkleProof[i];
    // Combine in canonical order to prevent tree structure attacks
    if (leaf_index & (1 << i) == 0) {
      current = persistentHash<Vector<2, Field>>([current, proofElement]) as Field;
    } else {
      current = persistentHash<Vector<2, Field>>([proofElement, current]) as Field;
    }
  }

  assert(current == merkleRoot, "Merkle proof failed");
}

3: Proof Aggregation

When you have multiple privacy preserving properties to prove, you have two choices: prove them all in one circuit (larger proof), or split into separate circuits (multiple proofs, sequential verification).

// Single circuit: proves age AND asset ownership
// Proof size: large, proving time: high
export circuit verifyAgeAndAssets(
  secret: Bytes<32>,
  minAge: Uint<32>,
  assetsProof: AssetsProof
): [] {
  const birthDate = getUserBirthDate();
  assert(currentTime - birthDate >= minAge, "Too young");

  const assets = getAssets(assetsProof);
  assert(assets.value >= 100000, "Insufficient assets");
}

// Split circuits: separate concerns, compose on app level
// Proof size: smaller per circuit, proving time: faster
export circuit verifyAge(secret: Bytes<32>, minAge: Uint<32>): Bytes<32> {
  const birthDate = getUserBirthDate();
  assert(currentTime - birthDate >= minAge, "Too young");
  return disclose(persistentHash<Bytes<32>>(secret));
}

export circuit verifyAssets(assetsProof: AssetsProof): Bytes<32> {
  const assets = getAssets(assetsProof);
  assert(assets.value >= 100000, "Insufficient assets");
  return disclose(persistentHash<Bytes<32>>(assetsProof));
}

// Application composes both proofs
// Tradeoff: two proofs to verify, but faster to prove each one

Benchmarking Your Circuits

Different circuit structures have dramatic performance differences. Use this framework to evaluate:

Structure	Pros	Cons	Use When
Direct computation	Simple, straightforward	Large proof, slow	Small bounded operations
Merkle proof verification	Logarithmic size, scales well	Higher cryptographic complexity	Membership checks in large sets
Vectorized batching	Efficient for repeated ops	Requires uniform structure	Batch processing many similar items
Split circuits	Faster per-circuit proving	Coordination overhead	When proofs are logically independent

Multi-Contract Privacy Architecture

Composing Private Contracts

Midnight's biggest strength is that multiple private contracts can interact while maintaining privacy boundaries. However, composing contracts introduces new considerations:

Problem: The Metadata Leak

Even if all data is encrypted, metadata can leak information:

// Privacy leak: contract call pattern reveals intent
export circuit buyAsset(assetId: Uint<64>): Bytes<32> {
  // If specific assetIds always correlate with specific users,
  // blockchain analysis can link buyers to assets even without seeing amounts
  const proof = getOwnershipProof(assetId);
  verify(proof);
  return disclose(persistentHash<Uint<64>>(assetId));
}

// Mitigated: hide specific asset, batch with dummy calls
export circuit batchBuyAssets<#N>(
  assetIds: Vector<#N, Uint<64>>,
  proofs: Vector<#N, Bytes<32>>,
  isReal: Vector<#N, Boolean>
): Vector<#N, Bytes<32>> {
  let results: Vector<#N, Bytes<32>> = [];
  for (let i = 0; i < #N; i++) {
    // Verify proof only if real purchase (circuits execute either way)
    if (isReal[i]) {
      verify(proofs[i]);
    }
    results[i] = disclose(persistentHash<Uint<64>>(assetIds[i]));
  }
  return results;
}

Pattern: Shielded Contract Composition

When one private contract depends on another, you need a protocol for safe interaction:

pragma language_version 0.22;
import CompactStandardLibrary;

// Contract A: Identity registry
export ledger identityCommitment: Map<Bytes<32>, Field>;

export circuit registerIdentity(publicKey: Bytes<32>): [] {
  const commitment = persistentHash<Bytes<32>>(publicKey) as Field;
  identityCommitment = disclose(publicKey, commitment);
}

// Contract B: Private voting (depends on Contract A)
export ledger voteCommitment: Map<Field, Field>; // (identityHash, voteHash)

export circuit castVote(
  publicKey: Bytes<32>,
  voteChoice: Uint<8>,
  salt: Bytes<32>
): [] {
  // Prove participation in Contract A without revealing identity
  const commitment = persistentHash<Bytes<32>>(publicKey) as Field;
  assert(identityCommitment[publicKey] == commitment, "Not registered");

  // Cast vote privately
  const voteHash = persistentHash<Vector<2, Field>>(
    [persistentHash<Uint<8>>(voteChoice) as Field, 
     persistentHash<Bytes<32>>(salt) as Field]
  ) as Field;

  voteCommitment = disclose(commitment, voteHash);
}

Key insight: Contract B proves it respects Contract A's invariants (the user is registered) without revealing the user's identity. This is the foundation of composable privacy.

Considerations: State Consistency

When multiple contracts touch shared state, you must be careful about ordering:

// Race condition possible: State updated after verification
export circuit transferWithFeeShare(
  amount: Uint<128>,
  feeRecipient: Bytes<32>
): [] {
  const balance = getBalance(); // Witness
  assert(balance >= amount + fee, "Insufficient");

  // Race condition: if another proof updates fee rate before on-chain execution,
  // this assertion might have been based on stale assumptions
  const currentFee = feeContract.queryFee();
}

//  Fixed: Include fee data in the proof
export circuit transferWithFeeShare(
  amount: Uint<128>,
  feeRecipient: Bytes<32>,
  expectedFeeRate: Uint<16>, // App provides expected fee
  feeProof: Bytes<32> // Proof that fee rate matches
): [] {
  const balance = getBalance();

  // Verify fee was what we expected at proof time
  verify(feeProof);

  const fee = (amount * expectedFeeRate) / 100000;
  assert(balance >= amount + fee, "Insufficient");
}

Real-World Tradeoffs

Decision Matrix

Use Case	Pattern	Witness Source	Proof Strategy	Notes
Privacy-first tokens	Commitments + Nullifiers	User secret key	Split by operation type	Requires nullifier tracking
KYC/AML compliance	Age/identity verification	Credential issuer	Selective disclosure	Issuer must be trusted
DAO voting	Shielded voting + identity registry	User secret, registry contract	Batched dummy votes	Metadata still visible (voting time)
Asset swaps	DEX with private pricing**	Oracle feeds, user orders	Batch matching	Requires MEV-resistant ordering

Debugging & Testing Advanced Patterns

Console Logging in Compact

While developing, use logging carefully (it's not available in production proofs):

export circuit debugTransfer(
  recipient: Bytes<32>,
  amount: Uint<128>
): [] {
  const balance = getBalance();

  // Debug logging helps during development
  assert(balance >= amount, "Insufficient");

  // The proof doesn't include debug output, but your app runner sees it
}

Testing Strategy for Circuits

Since circuits must be proven, test thoroughly before deployment:

// TypeScript test harness
import { Contract } from '@midnight-protocol/sdk';

const contract = new Contract(compiledCircuit);

// Test 1: Valid transfer
const validTransfer = await contract.transfer({
  recipient: publicKey,
  amount: 1000n,
  salt: randomSalt,
  newSalt: newRandomSalt
});

assert(validTransfer.proof != null, 'Valid transfer should produce proof');

// Test 2: Insufficient balance should fail
const invalidTransfer = await contract.transfer({
  recipient: publicKey,
  amount: 999999999999n,
  salt: randomSalt,
  newSalt: newRandomSalt
});

assert(invalidTransfer.proof == null, 'Invalid transfer should not produce proof');

Conclusion

Advanced Compact patterns aren't just optimizations they're architectural decisions that shape what's possible in your application.

As you move from learning Compact to building production systems, keep these principles in mind:

Witnesses are your privacy boundary. Choose witness sources carefully; they determine your security model.
Commitments enable privacy at scale. Direct state disclosure doesn't mix with zero-knowledge proofs; use commitments instead.
Circuits must be bounded, but cleverly. Vectorization, Merkle trees, and proof aggregation let you handle complexity without exceeding bounds.
Composition requires explicit coordination. Multiple private circuits can interact, but metadata and state consistency need careful handling.
Privacy and usability are in tension. Batching dummy transactions protects metadata but increases proof sizes. Splitting circuits proves faster but requires coordination. Choose based on your threat model.

Midnight gives you powerful tools for building privacy-first applications. Mastering these patterns lets you use them effectively.

DEV Community