DEV Community

Tosh
Tosh

Posted on

Bringing External Data On-Chain: Oracle Patterns for Midnight

#midnight #zk #blockchain #tutorial

Bringing External Data On-Chain: Oracle Patterns for Midnight

Midnight doesn't have a native oracle system. There's no Chainlink integration, no price feed contracts, no protocol-level mechanism for importing external state. If your contract needs the current ETH/USD price, a user's credit score, or the outcome of an off-chain event, you have to build the data pipeline yourself.

This isn't a limitation so much as a design consequence. Midnight's privacy model forces a harder question than most blockchains: who should know this data, and when? A public oracle feed that anyone can read is fine on Ethereum. On Midnight, public means permanently visible to all network participants — and that might not be what you want.

This article covers three oracle patterns, the trust tradeoffs in each, and TypeScript service code you can actually use.

Why Midnight Has No Native Oracle

On a standard EVM chain, oracle contracts work by storing data in contract state and letting other contracts read it. Chainlink nodes submit price updates in transactions; consumers call latestRoundData(). Simple.

Midnight's execution model breaks this in two ways.

First, circuit execution is local. When a user runs a Midnight circuit, it executes on their machine (or their DApp's server) before a ZK proof is generated and submitted. Circuits don't execute on nodes — nodes only verify proofs. So there's no on-chain execution context where a contract can "call" another to get live data.

Second, ledger state is public. Any data written to a Midnight contract's ledger is permanently visible on-chain. If you're building a privacy-preserving application and you need external data, you have to think carefully about whether that data should be public at all — and whether publishing it on-chain breaks the privacy properties you're trying to achieve.

That said, external data pipelines are entirely possible. They just require you to pick a pattern that matches your trust model.

Pattern 1: Witness-Provided Data

The prover (your user) supplies external data directly as a witness. This data is used inside the circuit — potentially to drive contract logic — but never appears on-chain directly.

When to use it: When the data is personal to the user, when you want the user to control their own data source, or when the data should stay private.

Example: User proves a balance threshold

Say you're building an age-gated service that also requires users to prove they hold at least $1,000 in an external account. You don't want to put the balance on-chain. You want the user to prove it privately.

contract GatedAccess {
  ledger grantedAccess: Set<Bytes<32>>;  // nullifiers for granted users

  witness getBalanceAttestation(): [Uint<64>, Bytes<32>, Bytes<64>] {
    // Returns: [balance, userId, signatureFromAttestation]
    // Implemented in TypeScript — calls an attestation service
  }

  export circuit proveAndGainAccess(public threshold: Uint<64>): [] {
    const [balance, userId, sig] = getBalanceAttestation();

    // Verify the attestation signature
    const attesterPubKey = bytes("0xATTESTER_PUBLIC_KEY");
    const msg = hash(userId, balance);
    assert verifySignature(attesterPubKey, msg, sig) : "invalid attestation";

    // Verify balance meets threshold
    assert balance >= threshold : "insufficient balance";

    // Grant access via nullifier (prevents replay)
    const nullifier = disclose(hash(userId, bytes("gated-access")));
    assert !ledger.grantedAccess.member(nullifier) : "already granted";
    ledger.grantedAccess = ledger.grantedAccess.insert(nullifier);
  }
}
Enter fullscreen mode Exit fullscreen mode

The TypeScript witness implementation fetches an attested balance from a service that signs the response:

const result = await contract.circuits.proveAndGainAccess(
  BigInt(1000_00),  // $1,000 threshold in cents
  {
    getBalanceAttestation: async (context) => {
      // Call an attestation service that verifies the user's account
      // and returns a signed response
      const response = await fetch("https://attest.example.com/balance", {
        method: "POST",
        headers: { Authorization: `Bearer ${userToken}` },
        body: JSON.stringify({ userId: context.privateState.userId })
      });

      const { balance, userId, signature } = await response.json();
      return [BigInt(balance), Buffer.from(userId), Buffer.from(signature, "hex")];
    }
  }
);
Enter fullscreen mode Exit fullscreen mode

Trust tradeoff: The user controls their data source. They could pass a fake attestation if the signature isn't verified in the circuit. The attestation service is the trust anchor — you're trusting it to sign only real balances.

What's public: That some anonymous user gained access (the nullifier). Not their balance, not their identity.

Pattern 2: Admin-Updated Ledger Fields

An authorized party publishes data to your contract's ledger in a separate transaction. Other circuits in the same contract (or other contracts reading public ledger state) consume it.

When to use it: When you need a canonical, consensus-verified data point that all users can rely on, and when transparency about the data's source is acceptable.

Example: Price oracle contract

contract PriceOracle {
  ledger prices: Map<Bytes<4>, Uint<64>>;    // assetId → price in cents
  ledger updatedAt: Map<Bytes<4>, Uint<64>>; // assetId → timestamp
  ledger owner: Bytes<32>;                   // oracle operator public key

  witness getAdminSignature(): [Bytes<64>] { ... }

  export circuit updatePrice(
    public assetId: Bytes<4>,
    public newPrice: Uint<64>,
    public timestamp: Uint<64>
  ): [] {
    // Verify the caller is the authorized oracle operator
    const [sig] = getAdminSignature();
    const msg = hash(assetId, newPrice, timestamp);
    assert verifySignature(ledger.owner, msg, sig) : "unauthorized";

    // Verify the update is more recent than the last
    const lastUpdate = ledger.updatedAt.lookup(assetId).value;
    assert timestamp > lastUpdate : "stale update";

    // Write to ledger (public, on-chain)
    ledger.prices = ledger.prices.insert(assetId, newPrice);
    ledger.updatedAt = ledger.updatedAt.insert(assetId, timestamp);
  }

  export circuit getPrice(public assetId: Bytes<4>): [Uint<64>] {
    const entry = ledger.prices.lookup(assetId);
    assert entry.isSome : "price not available";
    return [entry.value];
  }
}
Enter fullscreen mode Exit fullscreen mode

The TypeScript oracle feeder service that submits price updates:

import Anthropic from "@anthropic-ai/sdk";

// Price feeder that runs on an interval
async function feedPrices(
  oracleContract: Contract,
  adminWallet: WalletProvider,
  assets: string[]
): Promise<void> {
  for (const assetId of assets) {
    const price = await fetchPrice(assetId);
    const timestamp = BigInt(Math.floor(Date.now() / 1000));

    await oracleContract.circuits.updatePrice(
      Buffer.from(assetId),
      price,
      timestamp,
      {
        getAdminSignature: async (context) => {
          const msg = hashValues(
            Buffer.from(assetId),
            price,
            timestamp
          );
          const sig = adminWallet.sign(msg);
          return [sig];
        }
      }
    );

    console.log(`Updated ${assetId}: ${price} at ${timestamp}`);
  }
}

async function fetchPrice(assetId: string): Promise<bigint> {
  const response = await fetch(
    `https://api.coingecko.com/api/v3/simple/price?ids=${assetId}&vs_currencies=usd`
  );
  const data = await response.json();
  return BigInt(Math.round(data[assetId].usd * 100));  // cents
}

// Run every 60 seconds
setInterval(() => feedPrices(contract, wallet, ["bitcoin", "ethereum"]), 60_000);
Enter fullscreen mode Exit fullscreen mode

A consumer contract can read directly from the oracle's public ledger:

contract LendingPool {
  ledger priceOracleAddress: Bytes<32>;
  ledger collateral: Map<Bytes<32>, Uint<64>>;
  ledger borrowed: Map<Bytes<32>, Uint<64>>;

  export circuit checkSolvency(public userId: Bytes<32>): [Boolean] {
    // Read collateral and borrowed from this contract's ledger
    const userCollateral = ledger.collateral.lookup(userId).value;
    const userBorrowed = ledger.borrowed.lookup(userId).value;

    // Read BTC price from oracle's public ledger
    const oracle = getContractLedger(ledger.priceOracleAddress);
    const btcPriceEntry = oracle.prices.lookup(bytes("XBTC"));
    assert btcPriceEntry.isSome : "BTC price not available";
    const btcPrice = btcPriceEntry.value;

    // Collateral must be worth at least 150% of borrowed amount
    const collateralValue = userCollateral * btcPrice;
    const requiredValue = userBorrowed * 150 / 100;

    return [collateralValue >= requiredValue];
  }
}
Enter fullscreen mode Exit fullscreen mode

Trust tradeoff: All users rely on the oracle operator to publish accurate data. The operator is the single point of trust — they could publish wrong prices, or go offline. You gain consensus (all nodes agree on the price) but lose decentralization.

What's public: All price data and update timestamps. Anyone watching the chain can see every price update, when it was submitted, and derive the oracle's publishing pattern.

Pattern 3: Merkle-Attested Batch Data

The oracle publishes a Merkle root on-chain (one cheap transaction), and users prove membership of specific data points using Merkle proofs in witnesses. This is efficient when you need to make many data points queryable without publishing all of them.

When to use it: When you have large datasets (e.g., 10,000 eligible accounts, thousands of price points), when you want to minimize oracle update transactions, or when individual data points should stay private.

contract MerkleOracle {
  ledger dataRoot: Bytes<32>;   // Merkle root of the current dataset
  ledger rootTimestamp: Uint<64>;
  ledger owner: Bytes<32>;

  witness getDataWithProof(key: Bytes<32>): [
    Uint<64>,          // value
    Bytes<32>[20],     // Merkle siblings
    Boolean[20]        // path bits (left=false, right=true)
  ] { ... }

  export circuit verifyDataPoint(
    public key: Bytes<32>,
    public expectedValue: Uint<64>
  ): [Boolean] {
    const [value, siblings, pathBits] = getDataWithProof(key);

    // Compute leaf commitment
    const leaf = hash(key, value);

    // Reconstruct Merkle root from proof
    let node = leaf;
    for (let i = 0; i < 20; i++) {
      const sibling = siblings[i];
      node = pathBits[i]
        ? poseidon(sibling, node)
        : poseidon(node, sibling);
    }

    // Verify against published root
    assert node == ledger.dataRoot : "invalid proof";

    // Verify value matches expectation (can be relaxed to range check)
    return [value == expectedValue];
  }
}
Enter fullscreen mode Exit fullscreen mode

The oracle operator publishes only the root:

async function publishMerkleRoot(
  dataset: Map<string, bigint>,
  oracleContract: Contract,
  adminWallet: WalletProvider
): Promise<void> {
  // Build Merkle tree from dataset
  const tree = new MerkleTree(20);
  for (const [key, value] of dataset.entries()) {
    const leaf = poseidonHash([Buffer.from(key), bigIntToBytes(value)]);
    tree.insert(leaf);
  }

  const root = tree.root();
  const timestamp = BigInt(Math.floor(Date.now() / 1000));

  // One transaction publishes the root for the entire dataset
  await oracleContract.circuits.updateRoot(root, timestamp, {
    getAdminSignature: async () => [adminWallet.sign(hashValues(root, timestamp))]
  });

  console.log(`Published root for ${dataset.size} entries`);
}

// Users get proofs from an off-chain proof server
async function getProofForKey(key: string): Promise<MerkleProof> {
  const response = await fetch(`https://oracle-proofs.example.com/proof/${key}`);
  return response.json();
}
Enter fullscreen mode Exit fullscreen mode

Trust tradeoff: The oracle operator still controls the dataset. But users can verify that their specific data point is included in the committed root — they don't have to trust that the full dataset is correct, only that their element is in the tree.

What's public: Only the Merkle root and timestamp. Individual data points stay off-chain until a user proves membership.

Choosing the Right Pattern

Criterion Witness-Provided Admin Ledger Merkle Batch
Data is personal to user Yes No Possible
Need on-chain consensus No Yes Yes (root only)
Many data points No No Yes
Data should be private Yes No Partially
Freshness per-user Yes Oracle-controlled Oracle-controlled
On-chain cost Low High (one tx per update) Low (one tx per batch)
Trust anchor User's data source Oracle operator Oracle operator

Use witness-provided data when users control their own data (balances, credentials, identity attributes). The user's attestation service is the trust anchor. This pattern gives you the strongest privacy properties.

Use admin-updated ledger when you need canonical, public, consensus-verified data that all users read from the same source. Price feeds for DeFi applications are the obvious case. The oracle operator is trusted; transparency about updates is a feature.

Use Merkle batch when you have large datasets and want to minimize oracle transactions, or when you want data points to stay off-chain until proven. Eligibility lists, large price datasets, and compliance allowlists work well here.

Running a Production Oracle Feeder

For a production admin-ledger oracle, you need three things: the feeder service, a reliable data source, and monitoring.

import cron from "node-cron";

class OracleFeeder {
  constructor(
    private contract: Contract,
    private wallet: WalletProvider,
    private dataSources: DataSource[]
  ) {}

  async updateAll(): Promise<void> {
    const results = await Promise.allSettled(
      this.dataSources.map(source => this.updateOne(source))
    );

    const failed = results.filter(r => r.status === "rejected");
    if (failed.length > 0) {
      console.error(`${failed.length} updates failed:`, failed);
    }
  }

  private async updateOne(source: DataSource): Promise<void> {
    const value = await source.fetch();
    const timestamp = BigInt(Math.floor(Date.now() / 1000));

    await this.contract.circuits.updatePrice(
      source.assetId,
      value,
      timestamp,
      {
        getAdminSignature: async (_ctx) => {
          const msg = poseidonHash([source.assetId, value, timestamp]);
          return [this.wallet.sign(msg)];
        }
      }
    );
  }
}

// Run every 30 seconds
const feeder = new OracleFeeder(contract, adminWallet, priceSources);
cron.schedule("*/30 * * * * *", () => feeder.updateAll());
Enter fullscreen mode Exit fullscreen mode

Two operational concerns that matter in practice:

Staleness handling. Your contracts should check the updatedAt timestamp and reject decisions based on data older than some threshold. An oracle that goes offline shouldn't cause your lending protocol to silently use a week-old price.

Key management. The oracle admin key is high-value — anyone who compromises it can publish arbitrary data. Keep it in a hardware security module or use a threshold signing scheme if you're running this in production.

A Note on Privacy

Oracle data that goes on-chain is public forever. This seems obvious but has non-obvious implications.

If your oracle publishes BTC prices every 30 seconds, on-chain observers can reconstruct the full price history with timestamps — and potentially correlate oracle update patterns with the oracle operator's behavior. That might be fine for a price feed. It's probably not fine for an oracle that publishes private user data "for the user's own contract."

Witness-based oracles avoid this entirely. No data hits the chain. The user's witness is consumed by the circuit, the ZK proof commits to the fact (without revealing the value), and only the proof goes on-chain.

If you're building anything where the oracle data is sensitive — health records, financial history, identity attributes — use witnesses, not ledger state.

Top comments (0)