DolphinScheduler 3.1.9 Dev Setup Guide for Custom Development

In production, Apache DolphinScheduler 3.1.9 is widely chosen by enterprises for its stability and mature features. As usage deepens, many want to extend and customize it further. Before diving into secondary development, a complete local development environment setup is essential. This guide uses real-world scenarios to detail the configuration process for DolphinScheduler 3.1.9, aiming to be a practical and reliable reference for users with customization needs.

Prerequisites

Before setting up the DolphinScheduler dev environment, please ensure you have installed:

• Git
• JDK v1.8.x (Note: DolphinScheduler 3.1.9 requires JDK 1.8.x, no higher)
• Maven v3.5+
• Node v16.13+ (For DolphinScheduler versions earlier than 3.0, install Node v12.20+)
• Pnpm v6.x (Note: DolphinScheduler 3.1.9 requires Pnpm 6.x, no higher)

Open pom.xml

Open the pom.xml file using IntelliJ IDEA as a project.
In the .idea directory, locate workspace.xml, and under PropertiesComponent, add the line:

"dynamic.classpath": "true"

Download & Extract ZooKeeper

In the ZooKeeper folder, create a data directory.
Copy zoo_sample.cfg from conf/ and rename it to zoo.cfg. Change the dataDir path to point to the newly created data directory.

Start ZooKeeper by running:

./bin/zkServer.cmd

Modify DB Dependency Configuration

Open the pom.xml in the dolphinscheduler-bom module.
If you're using PostgreSQL, no changes are needed. For MySQL, change the dependency scope from test to compile, then refresh Maven in IntelliJ.

Create & Initialize the Database

Install MySQL 8.0.16+ (recommended).
After creating the database, initialize it using the SQL script files as shown in the screenshot.

Configure MySQL Settings

In the following modules' application.yaml and logback-spring.xml, change the database settings from PostgreSQL to MySQL and adjust log levels:

• dolphinscheduler-alert
• dolphinscheduler-api
• dolphinscheduler-master
• dolphinscheduler-worker

For example, in dolphinscheduler-master:

Also update logging in logback-spring.xml:

Adjust Frontend Module Settings

Open dolphinscheduler-ui/package.json and remove all ^ from version numbers to avoid conflicts:

Start Backend & Frontend Services

You need to start these three backend services:

• MasterServer: Run org.apache.dolphinscheduler.server.master.MasterServer::main in IntelliJ with VM options:

-Dlogging.config=classpath:logback-spring.xml 
-Ddruid.mysql.usePingMethod=false 
-Dspring.profiles.active=mysql

• WorkerServer: Run org.apache.dolphinscheduler.server.worker.WorkerServer::main with VM options:

-Dlogging.config=classpath:logback-spring.xml 
-Ddruid.mysql.usePingMethod=false 
-Dspring.profiles.active=mysql

• ApiApplicationServer: Run org.apache.dolphinscheduler.api.ApiApplicationServer::main with VM options:

-Dlogging.config=classpath:logback-spring.xml 
-Dspring.profiles.active=api,mysql

Open your run configurations, add the VM options, and hit Run.

Start Frontend

In the dolphinscheduler-ui directory, run:

pnpm install
pnpm dev

Once started, the terminal will display the port. Open your browser to access the web UI.

🎉 Once done, your DolphinScheduler 3.1.9 environment is fully up and ready for customization and development!