DEV Community

Chen Debra
Chen Debra

Posted on

Apache DolphinScheduler Standalone Deployment with Docker: A Practical 10-Minute Guide

📝 Overview: This hands-on guide walks you through deploying DolphinScheduler in Standalone mode with Docker. We’ll first compare the trade-offs between Standalone, Pseudo-Cluster, and Cluster modes, then build the environment in four steps: deploy MySQL and replace the default H2 database, initialize the official database schema, mount the MySQL JDBC driver, and launch everything with Docker Compose.

Along the way, we’ll cover three common pitfalls: you can’t use 127.0.0.1 to connect to a host database from inside a container, switching to MySQL does not automatically create the required tables, and failing to configure TZ can shift scheduled jobs by a full eight hours. With MySQL persistence in place, you can rebuild DolphinScheduler whenever needed, avoid data migration, and upgrade to a clustered deployment later with minimal effort. This setup is a good fit for small and mid-sized teams running offline data warehouses. Still using crontab to schedule your data warehouse jobs? Finding out that a job failed only after a colleague tells you about it? It may be time for a proper workflow scheduling platform. This guide shows you how to deploy DolphinScheduler in Standalone mode with Docker in about 10 minutes—lightweight, practical, and with a monitoring setup included.

Which DolphinScheduler Deployment Mode Should You Choose?

DolphinScheduler offers three deployment modes. Spend 30 seconds understanding the differences now, and you’ll save yourself a lot of rework later (don’t ask how I know 🥲):

Mode Key Characteristics Recommended For
Standalone All services run in a single process, with an embedded ZooKeeper and H2 database. Ready to use out of the box. Quick evaluation, testing, and small to mid-sized teams
Pseudo-Cluster All services are deployed on a single machine, but components such as Master, Worker, and API Server run as separate processes. Single-host environments that require more granular control
Cluster Services are distributed across multiple machines, with Master and Worker nodes supporting horizontal scaling. Large-scale production environments

In short: For a small team, Standalone mode is usually enough. And if you replace the default database with MySQL as described in this guide, you can later move to a clustered deployment without migrating your data (we’ll cover this in Section 6).

2. Why Choose Standalone Mode?

There are plenty of cluster deployment tutorials online, but they can look intimidating—three machines to start with, a ZooKeeper cluster, MySQL primary-replica setups, and so on.

But hold on. Do you really need all that complexity for your use case?

Here’s what my actual workload looks like:

  • 🔢 25 workflows, generating fewer than 300 workflow instances per day
  • 📊 Fewer than 1,500 task instances per day
  • ✅ Standalone mode runs reliably without any noticeable pressure

More importantly, in an offline data warehouse environment, even if DolphinScheduler goes down briefly, there is no direct impact on the online business. If a task runs a few minutes late, no data is lost and the world doesn’t end.

💡 The key idea: Start with Standalone mode, but replace the default H2 database with MySQL.

This is important because Standalone mode uses jdbc:h2:mem: by default. That means the database is purely in memory. It’s not a case of “you might lose some data.” Once the container restarts, workflow definitions, scheduling records, and other data are reset completely. Running production workloads on it is basically like keeping your valuables in a cabinet that gets emptied every time the power goes out.

With MySQL, your data becomes persistent, which gives you much greater peace of mind.

There’s another major benefit: once the data is stored in MySQL, DolphinScheduler itself can be rebuilt whenever necessary. If you later need additional plugins, a custom image, or a different deployment setup, simply redeploy the containers and point them to the same MySQL database. Your existing configuration comes back with it, so you can start over without rebuilding everything from scratch.

3. Hands-On Deployment in Four Steps

Step 1: Deploy MySQL

If you already have a MySQL instance available, you can skip this step and go straight to creating the database and granting permissions.

docker run --name dolphin-mysql \
  -e MYSQL_ROOT_PASSWORD='YourRootPass@2025' \
  -e MYSQL_DATABASE=ds_scheduler \
  -e MYSQL_USER=ds_admin \
  -e MYSQL_PASSWORD='DsPass@2025' \
  -p 13306:3306 \
  -v /data/dolphin-mysql/data:/var/lib/mysql \
  --restart unless-stopped \
  -d mysql:8.0.42 \
  --default-authentication-plugin=mysql_native_password \
  --character-set-server=utf8mb4 \
  --collation-server=utf8mb4_unicode_ci
Enter fullscreen mode Exit fullscreen mode

A few things to keep in mind:

  • The port mapping uses 13306 instead of the default 3306 to avoid conflicts with an existing MySQL instance on the host.
  • mysql_native_password avoids the extra RSA public-key exchange required by caching_sha2_password for certain non-SSL connections. ⚠️ Version note: This option has been deprecated since MySQL 8.0.34. You may see a deprecation warning at startup, but it still works. MySQL 8.4 removes the plugin entirely. If you’re using MySQL 8.4 or later, remove this option and use caching_sha2_password instead. The MySQL 8.x JDBC driver supports it.
  • Make sure you change the passwords to your own strong passwords. Don’t do what I did before and use 123456 (true story, but let’s not go there).

Step 2: Initialize the Database Schema

After the MySQL container starts, it will create an empty database, but it will not automatically create the dozens of tables required by DolphinScheduler. You need to manually execute the official initialization SQL script:

# Download the DolphinScheduler 3.2.0 MySQL initialization script
wget -O /tmp/dolphinscheduler_mysql.sql \
  https://raw.githubusercontent.com/apache/dolphinscheduler/3.2.0/dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_mysql.sql

# Copy the SQL file into the MySQL container
docker cp /tmp/dolphinscheduler_mysql.sql dolphin-mysql:/tmp/

# Initialize the database (replace the password with your own)
docker exec -i dolphin-mysql \
  mysql -uds_admin -p'DsPass@2025' ds_scheduler \
  < /tmp/dolphinscheduler_mysql.sql
Enter fullscreen mode Exit fullscreen mode

After the script finishes, verify that the tables were created successfully:

docker exec dolphin-mysql \
  mysql -uds_admin -p'DsPass@2025' ds_scheduler \
  -e "SHOW TABLES;" | head -20
Enter fullscreen mode Exit fullscreen mode

Under normal circumstances, you should see tables such as t_ds_process_definition and t_ds_worker_group. If nothing is returned, check whether the SQL initialization command reported any errors.

⚡ Do not skip this step! After switching to MySQL, the required tables are not created automatically. The initialization that happens automatically with the default H2 setup does not happen here. If you skip this step, you’ll likely be greeted by errors such as Table 'xxx' doesn't exist when DolphinScheduler starts.

Step 3: Download the MySQL JDBC Driver

The DolphinScheduler Docker image does not include the MySQL JDBC driver by default—it can’t know which database you’re going to use. You therefore need to download the driver and mount it into the container:

mkdir -p /data/dolphin/lib

wget -P /data/dolphin/lib/ \
  https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.30/mysql-connector-java-8.0.30.jar
Enter fullscreen mode Exit fullscreen mode

🤔 Why not use the latest driver?

First, 8.0.30 has been verified to work with this setup. When it comes to operations, there’s a simple rule: if it works, don’t change it unless you have a reason to.

Second, starting with 8.0.31, MySQL officially changed the artifact name from mysql-connector-java to mysql-connector-j. The Maven coordinates changed from mysql:mysql-connector-java to com.mysql:mysql-connector-j. If you search for the latest version but continue using the old download path, you’ll simply get a 404. If you want to use a newer driver, update both the download URL and the filename used in the volume mount.

Step 4: Deploy DolphinScheduler

Create /data/dolphin/docker-compose.yml:

services:
  dolphinscheduler-standalone:
    image: apache/dolphinscheduler-standalone-server:3.2.0
    container_name: dolphinscheduler-standalone

    # network_mode: "host"
    # If you enable host networking, you must also comment out
    # both the extra_hosts and ports sections below.

    extra_hosts:
      - "host.docker.internal:host-gateway"
      # Required on Linux so the container can resolve this hostname

    ports:
      - "12345:12345"
      # Web UI + API port (the only port that must be mapped)
      - "25333:25333"
      # Python gateway port; only required if you use
      # PyDolphinScheduler to build workflows

    environment:
      - TZ=Asia/Shanghai
      - DATABASE_TYPE=mysql
      - SPRING_DATASOURCE_DRIVER_CLASS_NAME=com.mysql.cj.jdbc.Driver

      # IMPORTANT: Use the actual host IP or host.docker.internal here.
      # Do NOT use 127.0.0.1.
      - SPRING_DATASOURCE_URL=jdbc:mysql://host.docker.internal:13306/ds_scheduler?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true
      - SPRING_DATASOURCE_USERNAME=ds_admin
      - SPRING_DATASOURCE_PASSWORD=DsPass@2025

    volumes:
      - /data/dolphin/logs:/opt/dolphinscheduler/logs

      # Mount the MySQL JDBC driver into the container's libs directory
      - /data/dolphin/lib/mysql-connector-java-8.0.30.jar:/opt/dolphinscheduler/libs/standalone-server/mysql-connector-java-8.0.30.jar

    restart: always
Enter fullscreen mode Exit fullscreen mode

⏰ Don’t skip the TZ setting.

A workflow scheduling platform is all about scheduled jobs. If the container defaults to UTC and you configure a job to run at 2:00 AM Beijing time, it will actually run at 10:00 AM. This kind of bug doesn’t throw an obvious error or crash the system. It simply makes your data warehouse jobs run eight hours late—which can be much harder to troubleshoot than a straightforward database connection failure.

After configuring it, run date inside the container to verify the timezone. If it shows CST, the setting has taken effect.

Start the service:

docker compose up -d
Enter fullscreen mode Exit fullscreen mode

⚠️ Common pitfall:

The database URL must not use 127.0.0.1 when DolphinScheduler is running inside a container. From inside the container, 127.0.0.1 refers to the container itself, not the host machine.

Here are the correct options:

Option Example Notes
Use the host IP jdbc:mysql://192.168.1.100:13306/ds_scheduler The most universal option; works on Linux and macOS
Docker special hostname jdbc:mysql://host.docker.internal:13306/ds_scheduler Supported natively by Docker Desktop on macOS and Windows; on Linux, add extra_hosts: ["host.docker.internal:host-gateway"] to docker-compose.yml
Host networking mode Enable network_mode: "host" and use 127.0.0.1:13306 in the URL The simplest option, but you must comment out both the ports and extra_hosts sections. With host networking, the container shares the host's network stack, so port mapping is no longer used and host.docker.internal is unnecessary.

4. Verify the Deployment

After startup, wait about 30 seconds. The container needs some time to start the embedded ZooKeeper and the other services.

Then open the following URL in your browser:

http://<YOUR_SERVER_IP>:12345/dolphinscheduler/ui
Enter fullscreen mode Exit fullscreen mode

Log in with the default credentials:

Username Password
admin dolphinscheduler123

If you see the login page, the service is up and running:

🔐 Change the password immediately after logging in! A default password is basically an unlocked door. Don’t wait until someone walks through it to regret leaving it open.

Once you’ve logged in successfully and can see the home dashboard, the deployment is complete:

You can also create a simple Shell task and run:

echo "Hello DolphinScheduler"
Enter fullscreen mode Exit fullscreen mode

This is a quick way to verify that workflow scheduling is working as expected.

5. Don’t Forget Monitoring!

With a Standalone deployment, the biggest risk isn’t that the service goes down.

It’s that the service goes down and nobody notices.

We strongly recommend pairing the deployment with Prometheus + Grafana monitoring so you can:

  • 📈 Monitor task execution status in real time
  • 🚨 Trigger automatic alerts when something goes wrong, via Feishu, DingTalk, email, or other channels
  • 📊 Build visual dashboards to monitor the health of your scheduling platform at a glance

6. Future Upgrade: Move to Cluster Mode Without Migrating Your Data

You may be wondering: “If I start with Standalone mode and eventually outgrow it, how difficult will the migration be?”

The good news is that because we chose MySQL instead of the default H2 database from the beginning, all workflow definitions, task configurations, scheduling records, tenant information, and other persistent data are stored in MySQL.

When you later move to Pseudo-Cluster or Cluster mode:

  1. Reuse your existing data: The new Master, Worker, and API Server components only need to point to the same MySQL instance. They can then read all existing data, so there is no need to recreate workflows or scheduled jobs.
  2. Replacing ZooKeeper has no impact on business data: The embedded ZooKeeper in Standalone mode is responsible only for runtime coordination, including service registration and discovery, Master/Worker heartbeats, and distributed locks. It stores temporary nodes (ephemeral nodes), not business data. When you switch to an external ZooKeeper cluster, the services simply register themselves again when they start. Your workflows and historical data remain unaffected. It’s basically like moving to a new meeting room—everyone just checks in again.
  3. Deployment process: Stop the Standalone container → deploy an independent ZooKeeper cluster → deploy multiple service components in Cluster mode → connect them to the same MySQL instance and the new ZooKeeper cluster → start the services.
  4. No data migration required: There is no need to export and import data or run a migration script. The existing MySQL schema and data can be reused directly.

💡 Here’s an easy way to remember the division of responsibilities:

MySQL stores the “memory”; ZooKeeper stores the “state.”

Replacing ZooKeeper simply means establishing a new connection between the services. The memory stays intact.

And that is the deeper reason for replacing H2 with MySQL in the first place: it’s not just about data persistence and reliability; it also keeps the door open for future architecture upgrades.

7. Summary

Item Details
Deployment mode Docker Standalone mode
Database MySQL 8.0 replacing the default H2
Recommended scenarios Testing, small to mid-sized teams, fewer than 2,000 tasks/day
Deployment time About 10 minutes (excluding coffee breaks)
Key pitfall #1 You cannot use 127.0.0.1 to connect to the host's MySQL from inside the container
Key pitfall #2 Switching to MySQL does not automatically create the required tables; you must run the official initialization SQL first
Key pitfall #3 Without TZ, scheduled jobs will run according to UTC and can be shifted by a full eight hours

The takeaway: Don’t over-engineer it. For offline scheduling at a small or mid-sized organization, Standalone mode + MySQL + monitoring and alerting is often the best balance of simplicity, reliability, and operational cost.

And when the day finally comes that you genuinely need a cluster, you can scale up then.

Top comments (0)