DEV Community

Cover image for The Complete Guide to Running a Midnight Node: Setup, Sync & Monitoring
Uroy Nwankwo
Uroy Nwankwo

Posted on

The Complete Guide to Running a Midnight Node: Setup, Sync & Monitoring

#web3 #tutorial #blockchain #docker

If you've been building on Midnight or want to participate in the network without trusting third-party infrastructure, run your own full node. This tutorial takes you from a bare Ubuntu/Debian machine to a healthy, synced node, including common failure points such as dropped peers and getting stuck on block 1.

Target audience: Developers and node operators who are comfortable with the Linux command line and Docker.

Prerequisites:

  • A Ubuntu 22.04 or Debian 12 server (bare metal or VPS)
  • Docker Engine installed (covered below)
  • Git installed
  • A running Cardano-db-sync + PostgreSQL instance (covered in step 1)
  • Open firewall on port 30333 (P2P)

What you'll have by the end: A full node synced to Midnight's Testnet, with monitoring commands and troubleshooting steps for common issues.

Understanding the node types

Before you spin anything up, it helps to know what you're actually running. Midnight has four node types:

  • Full node: syncs the chain, validates transactions, and serves real-time state queries. It prunes historical state older than 256 blocks by default. This is what most developers and node operators need.
  • Archive node: the same as a full node but retains the entire chain history. Required for block explorers or deep historical queries. Uses more disk space.
  • RPC node: exposes an HTTP/WebSocket API for DApps to interact with the chain programmatically.
  • Boot node: helps new nodes discover peers. You don't need to run one unless you're contributing to network bootstrapping.

This tutorial focuses on the full node. Where the archive node diverges, this guide calls out the one flag you need to change.

Step 1: Set up Cardano-db-sync and PostgreSQL

Midnight operates as a Cardano Partnerchain. That means your Midnight node needs a live connection to a Cardano-db-sync instance to track relevant scripts on the Cardano blockchain. This is required. Your node will not function without it.

Clone the midnight-node-docker repository

The midnight-node-docker repository contains the official Docker Compose files for the entire stack.

git clone https://github.com/midnightntwrk/midnight-node-docker
cd midnight-node-docker
Enter fullscreen mode Exit fullscreen mode

Install Docker Engine

If Docker isn't already installed:

# Add Docker's official GPG key
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
  -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository
echo \
  "deb [arch=$(dpkg --print-architecture) \
  signed-by=/etc/apt/keyrings/docker.asc] \
  https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io \
  docker-buildx-plugin docker-compose-plugin
Enter fullscreen mode Exit fullscreen mode

Verify the installation:

docker --version
sudo systemctl status docker
Enter fullscreen mode Exit fullscreen mode

You should see Active: active (running) in the status output.

Configure the Cardano-db-sync service

Open compose-partner-chains.yml in an editor:

nano compose-partner-chains.yml
Enter fullscreen mode Exit fullscreen mode

Make the following changes:

  1. Replace all default usernames and passwords with values only you know. Do not leave postgres / password123 in a production or publicly reachable environment.
  2. Comment out the Ogmios service unless you are a block producer who needs it. Find the ogmios: block and add # to the start of each line.
  3. Save the file.

Start Cardano-db-sync 

docker compose -f compose-partner-chains.yml up -d
Enter fullscreen mode Exit fullscreen mode

This starts Cardano-db-sync and PostgreSQL as background services.

Verify Cardano-db-sync is syncing

Install the PostgreSQL client if you don't have it:

sudo apt-get install postgresql-client
Enter fullscreen mode Exit fullscreen mode

Connect and run the sync progress query:

psql -h localhost -U your_postgres_user -d cexplorer -p 5432
Enter fullscreen mode Exit fullscreen mode

Inside the PostgreSQL shell:

SELECT 100 * (
    EXTRACT(EPOCH FROM (MAX(time) AT TIME ZONE 'UTC')) -
    EXTRACT(EPOCH FROM (MIN(time) AT TIME ZONE 'UTC'))
) / (
    EXTRACT(EPOCH FROM (NOW() AT TIME ZONE 'UTC')) -
    EXTRACT(EPOCH FROM (MIN(time) AT TIME ZONE 'UTC'))
) AS sync_percent
FROM block;
Enter fullscreen mode Exit fullscreen mode

You'll see a number between 0 and 100. Cardano-db-sync must reach 100% before your Midnight node can operate correctly. On the Preview Testnet this typically takes several hours depending on your hardware and disk IOPS. Grab a coffee.

Tip: You can keep checking sync progress without entering the shell every time:

docker exec -it db-sync-postgres psql -U your_postgres_user \
  -d cexplorer -c \
  "SELECT 100 * (EXTRACT(EPOCH FROM (MAX(time) AT TIME ZONE 'UTC')) - \
  EXTRACT(EPOCH FROM (MIN(time) AT TIME ZONE 'UTC'))) / \
  (EXTRACT(EPOCH FROM (NOW() AT TIME ZONE 'UTC')) - \
  EXTRACT(EPOCH FROM (MIN(time) AT TIME ZONE 'UTC'))) \
  AS sync_percent FROM block;"

Step 2: Resource requirements

Before you start the Midnight node itself, make sure your machine meets the minimum specs. Running an undersized node is the fastest way to end up stuck on block 1 with peers constantly dropping.

Resource Full node (Preview Testnet) Archive node (Preview Testnet)
CPU 4 cores 4 cores
Memory 16 GB RAM 16 GB RAM
Storage 40 GB SSD (for Cardano-db-sync) + 20 GB for node data 150 GB+ SSD
Disk IOPS 30,000+ 30,000+
Network Stable connection, port 30333 open Stable connection, port 30333 open

Storage note: Low-IOPS disks, including spinning HDDs and cheap cloud volumes, often cause sync stalls and peer disconnections. If your node keeps falling behind the chain tip, check your disk I/O before anything else:

iostat -x 1 5
Enter fullscreen mode Exit fullscreen mode

Look at the %util column. If it stays above 80%, your disk is the bottleneck.

Step 3: Configure and start the full node

Check the current node version

Always use the version listed in the compatibility matrix. At the time of writing, the current node version is 0.22.3 on Preview and Preprod.

Create the Docker Compose file

Create a file called compose-midnight-node.yml in your working directory:

nano compose-midnight-node.yml
Enter fullscreen mode Exit fullscreen mode

Paste the following:

version: "3.8"

services:
  midnight-full-node:
    image: midnightnetwork/midnight-node:0.22.3
    container_name: midnight-full-node
    platform: linux/amd64
    restart: unless-stopped
    ports:
      - "30333:30333"
    volumes:
      - midnight-data:/node
    environment:
      MIDNIGHT_NODE_IMAGE: "midnightnetwork/midnight-node:0.22.3"
      POSTGRES_HOST: "your_postgres_host"
      POSTGRES_PORT: "5432"
      POSTGRES_USER: "your_postgres_user"
      POSTGRES_PASSWORD: "your_postgres_password"
      POSTGRES_DB: "cexplorer"
      DB_SYNC_POSTGRES_CONNECTION_STRING: "psql://your_postgres_user:your_postgres_password@your_postgres_host:5432/cexplorer"
      BASE_PATH: "./node/chain/"
      CFG_PRESET: "testnet-02"
    command: >
      --chain=/res/testnet-02/testnetRaw.json
      --no-private-ip

volumes:
  midnight-data:
Enter fullscreen mode Exit fullscreen mode

Replace all placeholder values (your_postgres_host, your_postgres_user, your_postgres_password) with the credentials you set in step 1.

If you want an archive node instead, add --pruning archive to the command block:

    command: >
      --chain=/res/testnet-02/testnetRaw.json
      --pruning archive
      --no-private-ip
Enter fullscreen mode Exit fullscreen mode

Start the node 

docker compose -f compose-midnight-node.yml up -d
Enter fullscreen mode Exit fullscreen mode

The node starts in the background. Give it 30–60 seconds to initialize, then check that it's running:

docker ps
Enter fullscreen mode Exit fullscreen mode

You should see midnight-full-node with status Up.

Step 4: Monitor the initial sync

Stream logs in real time 

docker logs -f midnight-full-node
Enter fullscreen mode Exit fullscreen mode

During a healthy sync, you'll see lines like:

2024-05-04 12:00:01 ⚙️  Syncing 45.2 bps, target=#187432 (12 peers), best: #8492 (0x1a2b…c3d4), finalized #8448 (0x5e6f…7a8b)
Enter fullscreen mode Exit fullscreen mode

Key things to watch:

  • bps: blocks per second being synced. A healthy node syncs at several dozen bps during initial sync.
  • target=#XXXXXX: the current chain tip.
  • best: #XXXXXX: your local best block. This should climb steadily.
  • finalized #XXXXXX: the last finalized block. This should trail your best block by a small number.
  • (N peers): the number of connected peers. You want to see 5 or more.

Monitor block height

To check your current block height at any point without reading the full log stream:

docker logs midnight-full-node 2>&1 | grep "best:" | tail -1
Enter fullscreen mode Exit fullscreen mode

Or query the node's RPC endpoint directly (if you've also started an RPC node):

curl -s -H "Content-Type: application/json" \
  -X POST \
  -d '{"jsonrpc":"2.0","id":1,"method":"chain_getHeader","params":[]}' \
  http://127.0.0.1:9944 | python3 -m json.tool
Enter fullscreen mode Exit fullscreen mode

The number field in the response is the current best block in hex. Convert it:

printf "%d\n" 0x2DB20   # example output: 187168
Enter fullscreen mode Exit fullscreen mode

Check peer count 

docker logs midnight-full-node 2>&1 | grep "peers" | tail -5
Enter fullscreen mode Exit fullscreen mode

A healthy node shows 5–50 peers. If you're consistently seeing 0–2 peers, see the troubleshooting section below.

Step 5: Verify your node is synced and healthy

Once you believe your node has caught up, confirm it's genuinely at the chain tip rather than just stopped at a stale block.

Check the sync gap 

docker logs midnight-full-node 2>&1 | grep "Syncing\|Idle" | tail -5
Enter fullscreen mode Exit fullscreen mode

When fully synced, the log line changes from Syncing to Idle:

2024-05-04 14:22:01 💤 Idle (14 peers), best: #187432 (0x1a2b…c3d4), finalized #187428 (0x5e6f…7a8b)
Enter fullscreen mode Exit fullscreen mode

Idle means your node is at the tip and waiting for new blocks. The best block number should match (or be within a few blocks of) the Midnight block explorer.

Compare with the network tip

You can check the current network block height against the Midnight block explorer. If your node's best block is within 5–10 blocks of the explorer, you're synced.

Verify P2P port is reachable

From another machine or using a port checking tool:

nc -zv your_server_ip 30333
Enter fullscreen mode Exit fullscreen mode

If the port is closed, your node can't accept inbound peer connections, which limits your peer count and sync speed. Open it in your firewall:

sudo ufw allow 30333/tcp
sudo ufw reload
Enter fullscreen mode Exit fullscreen mode

Step 6: Troubleshooting common issues

Node is stuck on block 1 (or a very low block number)

This is the most common issue reported in the community. There are three usual causes:

1. Cardano-db-sync hasn't finished syncing

Your Midnight node requires a fully synced Cardano-db-sync instance. If sync_percent in PostgreSQL is below 100, your node stalls. Check it with the query from step 1, and wait until it reaches 100%.

2. The PostgreSQL connection string is wrong

Double-check your DB_SYNC_POSTGRES_CONNECTION_STRING environment variable. A common mistake is using localhost as the host when PostgreSQL is running in a different Docker container. Use the actual container name or host IP instead.

Test the connection directly:

psql "psql://your_postgres_user:your_postgres_password@your_postgres_host:5432/cexplorer" \
  -c "SELECT count(*) FROM block;"
Enter fullscreen mode Exit fullscreen mode

If this fails, fix your connection string before restarting the node.

3. Node image version mismatch

Running an outdated node image against the current Testnet causes immediate sync failure. Always verify against the compatibility matrix and pull the latest image:

docker pull midnightnetwork/midnight-node:0.22.3
docker compose -f compose-midnight-node.yml up -d --force-recreate
Enter fullscreen mode Exit fullscreen mode

Peers keep disconnecting

If peer counts keep climbing to 5 and then dropping back to 0, check these items:

1. Port 30333 is not publicly reachable

This is the most common cause. Even if Docker is forwarding the port locally, your firewall or cloud security group might be blocking inbound connections. Peers connect to you on port 30333; if they can't reach it, they time out and disconnect.

Check your firewall:

sudo ufw status
Enter fullscreen mode Exit fullscreen mode

If 30333 isn't listed as ALLOW, add it:

sudo ufw allow 30333/tcp
sudo ufw reload
Enter fullscreen mode Exit fullscreen mode

If you're on a cloud VM (AWS, GCP, DigitalOcean), also check your provider's inbound security group or firewall rules in the web console.

2. The --no-private-ip flag is missing

Without --no-private-ip, your node advertises its private/internal IP address to peers. External peers try to connect to an address they can never reach, fail, and disconnect. Make sure this flag is in your command block.

3. NAT or double-NAT

If your server is behind NAT (common with home servers or some VPS setups), peers can't initiate connections to you. You either need to set up port forwarding at the router level, or use a VPS with a public IP.

4. Disk I/O bottleneck

High disk utilisation causes the node to fall behind processing blocks, which causes peers to consider it unresponsive and drop the connection. Check with:

iostat -x 1 10
Enter fullscreen mode Exit fullscreen mode

Upgrade to an SSD with 30,000+ IOPS if %util is consistently high.

Node crashes or restarts frequently

Check the logs for the last error before restart:

docker logs midnight-full-node 2>&1 | grep -E "ERROR|WARN|panic" | tail -20
Enter fullscreen mode Exit fullscreen mode

Common causes:

  • Out of memory: check with free -h. If your machine has less than 16 GB RAM, the node may OOM. Add swap as a temporary workaround: 
  sudo fallocate -l 8G /swapfile
  sudo chmod 600 /swapfile
  sudo mkswap /swapfile
  sudo swapon /swapfile
Enter fullscreen mode Exit fullscreen mode
  • Docker volume corruption: rare, but possible after a hard shutdown. Recreate the volume: 
  docker compose -f compose-midnight-node.yml down
  docker volume rm midnight-data
  docker compose -f compose-midnight-node.yml up -d
Enter fullscreen mode Exit fullscreen mode

Note: this wipes your sync progress and starts from block 0.

Checking connectivity manually

To confirm your node can actually talk to the network:

# Check if P2P port is listening
ss -tlnp | grep 30333

# Test reachability from outside (run on a different machine)
nc -zv your_server_ip 30333

# Check how many peer connections are currently open
docker exec midnight-full-node ss -tn | grep 30333 | wc -l
Enter fullscreen mode Exit fullscreen mode

Keeping your node up to date

Midnight is in active Testnet development. Node versions update frequently, and running an outdated version will cause your node to stop syncing or get rejected by peers.

Subscribe to the Midnight Discord channel on Discord and watch the release notes for new node versions.

When a new version drops, update like this:

# Pull the new image
docker pull midnightnetwork/midnight-node:NEW_VERSION

# Update the version in your compose file
nano compose-midnight-node.yml
# Change 0.22.3 to NEW_VERSION in both the image: and MIDNIGHT_NODE_IMAGE: lines

# Restart with the new image
docker compose -f compose-midnight-node.yml up -d --force-recreate
Enter fullscreen mode Exit fullscreen mode

What to do next

You now have a full node running, synced, and healthy. From here:

  • Set up an RPC node: if you want your DApp to talk directly to your own infrastructure instead of a public endpoint, follow the RPC node setup guide.
  • Explore the DApp quickstart: start building a privacy-preserving DApp on top of the network you're now running: Getting started.
  • Join the community: the Midnight developer forum and Discord are the best places to ask questions, report issues, and follow network updates.
  • Share your work: if you write about your node setup or share what you built on top of it, tag @midnightntwrk on X and use #MidnightforDevs.

This tutorial reflects Midnight node v0.22.3 on the Preview/Preprod Testnet. Always check the compatibility matrix before updating your setup.

Top comments (0)