DEV Community

Chen Debra
Chen Debra

Posted on

Apache DolphinScheduler Local Setup Made Simple: A Beginner-Friendly Guide

#beginners #opensource #apachedolphinscheduler #datascience

This article is intended for developers who want to read and debug the core source code of Apache DolphinScheduler locally. The example environment is based on Windows + IntelliJ IDEA + Docker Desktop + PostgreSQL + ZooKeeper.

If you only want to quickly تجربه features rather than debug the full chain of master / worker / api, it is recommended to use StandaloneServer first. If you want to debug the distributed scheduling workflow, follow this guide to start services separately.

Use Cases

  • Start MasterServer, WorkerServer, and ApiApplicationServer individually in IntelliJ IDEA
  • Use Docker Desktop to host PostgreSQL and ZooKeeper
  • Debug Java services locally on the host machine
  • Run the frontend locally and connect it to backend APIs

Environment Requirements

  • Docker Desktop
  • JDK 8 or 11
  • Maven 3.8+ (or use the built-in mvnw.cmd)
  • Node.js 16+
  • pnpm 8+
  • IntelliJ IDEA

The java.version in the root pom.xml is 1.8. It is recommended to use JDK 8 or 11 for local debugging.

1. Start PostgreSQL and ZooKeeper

First, navigate to the deploy/docker directory:

cd <your-path>\dolphinscheduler\deploy\docker
Enter fullscreen mode Exit fullscreen mode

If you are using the docker-compose-windows.yml provided in the appendix, ensure that dolphinscheduler-zookeeper exposes port 2181.

master, worker, and api all connect to localhost:2181 by default. If ZooKeeper runs only inside the container without port mapping, Java processes started in IDEA will fail to connect.

Ensure the following configuration exists:

dolphinscheduler-zookeeper:
  image: zookeeper:3.8
  ports:
    - "2181:2181"
Enter fullscreen mode Exit fullscreen mode

Start services:

docker-compose -f docker-compose-windows.yml up -d dolphinscheduler-postgresql dolphinscheduler-zookeeper
Enter fullscreen mode Exit fullscreen mode

Optional verification:

docker ps
Test-NetConnection 127.0.0.1 -Port 5432
Test-NetConnection localhost -Port 2181
Enter fullscreen mode Exit fullscreen mode

Expected results:

  • Port 5432 is reachable
  • Port 2181 is reachable

If you are using local or remote installations instead of Docker, skip this step but ensure configurations match your environment.

2. Build the Project 

cd <your-path>\dolphinscheduler
.\mvnw.cmd spotless:apply
.\mvnw.cmd clean install -DskipTests
Enter fullscreen mode Exit fullscreen mode

Notes:

  • spotless:apply formats code to avoid check failures
  • The first build may take a while

3. Initialize PostgreSQL Metadata Database

Before starting master and api, initialize metadata tables.

SQL script location:

dolphinscheduler-dao/src/main/resources/sql/dolphinscheduler_postgresql.sql
Enter fullscreen mode Exit fullscreen mode

Using Docker PostgreSQL:

Get-Content -Path .\dolphinscheduler-dao\src\main\resources\sql\dolphinscheduler_postgresql.sql -Raw |
docker exec -i -e PGPASSWORD=root docker-dolphinscheduler-postgresql-1 psql -U root -d dolphinscheduler
Enter fullscreen mode Exit fullscreen mode

Alternatively, use DataGrip, DBeaver, or psql.

Note: This script contains DROP TABLE IF EXISTS. Do NOT run it on production databases.

Verification:

select version from t_ds_version;
Enter fullscreen mode Exit fullscreen mode

Expected: one record returned (e.g., 3.4.0)

4. Verify Local Configuration

Default configs (usually no changes needed):

  • PostgreSQL: 127.0.0.1:5432
  • DB: dolphinscheduler
  • Username: root
  • Password: root
  • ZooKeeper: localhost:2181

Config files:

  • dolphinscheduler-master/.../application.yaml
  • dolphinscheduler-api/.../application.yaml
  • dolphinscheduler-worker/.../application.yaml

If needed, modify:

  • spring.datasource.url
  • spring.datasource.username
  • spring.datasource.password
  • registry.zookeeper.connect-string

Do NOT use:

-Dspring.profiles.active=mysql
Enter fullscreen mode Exit fullscreen mode

Use instead:

-Dspring.profiles.active=postgresql
Enter fullscreen mode Exit fullscreen mode

5. Configure IntelliJ IDEA Run Configurations

Common settings:

  • JDK: 8 or 11
  • Use the classpath of the module
  • Enable: Add dependencies with "provided" scope to classpath
  • Working directory: project root

This option is critical to avoid missing dependency issues.

Create these configurations:

MasterServer

  • Main class: org.apache.dolphinscheduler.server.master.MasterServer

Ports:

  • RPC: 5678
  • Spring Boot: 5679

WorkerServer

  • Main class: org.apache.dolphinscheduler.server.worker.WorkerServer

Ports:

  • RPC: 1234
  • Spring Boot: 1235

ApiApplicationServer

  • Main class: org.apache.dolphinscheduler.api.ApiApplicationServer

Ports:

  • HTTP: 12345
  • Gateway: 25333

Startup order:

  1. MasterServer
  2. WorkerServer
  3. ApiApplicationServer

6. Start Frontend 

cd <your-path>\dolphinscheduler\dolphinscheduler-ui
pnpm install
pnpm run dev
Enter fullscreen mode Exit fullscreen mode

Access:

http://localhost:5173
Enter fullscreen mode Exit fullscreen mode

Default credentials:

  • Username: admin
  • Password: dolphinscheduler123

7. Verification

API

  • /actuator/health → should return UP
  • /swagger-ui → should load successfully

Frontend

  • Access UI and log in successfully

Logs

Check for fatal errors in the IDEA console

8. Common Issues

ZooKeeper connection failed

  • ZooKeeper is not running
  • Port 2181 not exposed

Missing t_ds_version table

  • DB not initialized
  • Wrong database

Missing dependencies in IDEA

  • Check the “provided scope” option

Port 12345 occupied

  • Stop conflicting processes

Top comments (0)