How to Safely Upgrade Apache SeaTunnel 2.x Without Breaking Production

Once data integration enters steady production operation, upgrading Apache SeaTunnel is rarely a simple “upgrade whenever you want.” Version compatibility, configuration changes, and connector adjustments can all affect production workloads if handled improperly. Based on practical experience, this article provides a guide to upgrading to SeaTunnel 2.x to help you minimize upgrade risks.

1. Preparation Before Upgrading

1.1 Environment Check

• JDK Version: Confirm the JDK version supported by the new SeaTunnel release (usually JDK 8 or JDK 11 is recommended).
• Dependent Components: Check version compatibility with dependencies such as Hadoop, Spark, and Flink.

1.2 Backup (Critical)

Before starting the upgrade, you must back up your existing SeaTunnel installation directory and data.
Recommended backup items:

• Installation directory: The entire SeaTunnel installation package directory.

• Configuration files (config/):

  • seatunnel.yaml / seatunnel-env.sh
  • hazelcast.yaml (SeaTunnel Engine configuration file)
  • log4j2.properties (log configuration)

• Connectors and plugins (connectors/, plugins/): downloaded third-party JAR packages.

• Scripts (bin/): if you have modified startup scripts.

• Checkpoint data: If you enabled checkpoints, it is recommended to stop tasks and manually trigger a Savepoint as a backup before upgrading. However, please note that checkpoint/state data generated before version 2.3.12 is not compatible with newer versions. Therefore, if you attempt to restart using the -r parameter after the upgrade to recover from an old checkpoint, the job may fail to start directly. In most cases, it is recommended to restart jobs from scratch after upgrading. If necessary, you may try recovering from a Savepoint, but success is not guaranteed.

Example backup commands:

# Assume SeaTunnel is installed in /opt/seatunnel
# 1. Backup configuration files
cp -r /opt/seatunnel/config /opt/seatunnel/config_backup_$(date +%Y%m%d)

# 2. Or back up the entire directory (recommended)
tar -zcvf seatunnel_backup_$(date +%Y%m%d).tar.gz /opt/seatunnel

2. Download the New Version

2.1 Obtain the New Version

• Official Download: Visit the Apache SeaTunnel download page.

• Mirror and Alternative Sources: If the official source is slow or fails, you can use these alternatives:

  • Apache Archive (historical versions): https://archive.apache.org/dist/seatunnel/
  • Maven Central (binary packages): https://mvnrepository.com/artifact/org.apache.seatunnel
  • Tsinghua University mirror: https://mirrors.tuna.tsinghua.edu.cn/apache/seatunnel/

• Verify integrity: Download the .asc or .sha512 file to verify the integrity of the package and prevent file corruption.
  
  

# Example: download version 2.3.x
wget https://archive.apache.org/dist/seatunnel/2.3.x/apache-seatunnel-2.3.x-bin.tar.gz
tar -zxvf apache-seatunnel-2.3.x-bin.tar.gz

3. Migration and Configuration

3.1 Configuration File Migration

Do not directly overwrite configuration files in the new version. It is recommended to use the diff tool to compare old and new configuration files and migrate your custom settings into the new ones.

• seatunnel.yaml: Check JVM memory settings (jvm_options) and classpath configuration.
• hazelcast.yaml: Check network configuration (network) and cluster name (cluster-name). Ensure the cluster communication ports do not conflict between old and new versions (if running simultaneously).

3.2 Dependency Library Migration

Copy manually added third-party JAR files from the lib/ directory of the old version (such as JDBC drivers or Hadoop dependencies) into the lib/ directory of the new version.
Note: Check whether these JAR files conflict with the new SeaTunnel version.

4. Connector and Plugin Upgrade

Starting from SeaTunnel 2.3.0, connectors are decoupled from the engine.

4.1 Install New Connectors

Use the script provided in the distribution package to install required connectors:

cd apache-seatunnel-2.3.x
# View supported plugins
sh bin/install-plugin.sh --help
# Install specific connectors
sh bin/install-plugin.sh connector-cdc-mysql,connector-console

Alternatively, you can manually download the corresponding JAR packages from Maven Central into the connectors/seatunnel directory.

4.2 Compatibility Check

Make sure to check the connector compatibility matrix.

JDBC / StarRocks / Doris connectors
In most cases, the V2 Connector API maintains backward compatibility. However, you should verify:

• Driver version: The new connector may depend on a newer JDBC driver.
• Database version: Confirm the database versions supported according to the connector documentation.
• Configuration changes: Review the release notes to identify deprecated configuration items.

CDC connectors (MySQL / Postgres / Oracle, etc.)
CDC components are very sensitive to database transaction log formats. Before upgrading:

1. Confirm that the new CDC connector supports your database version.
2. Check whether server-side plugins need to be updated (such as Oracle LogMiner configuration or Postgres wal2json/decoderbufs).

Paimon connector compatibility example
Data lake components such as Paimon, Iceberg, and Hudi rely heavily on specific file formats and APIs, so version requirements are strict.

SeaTunnel Version	Paimon Version
2.3.2 – 2.3.3	0.4-SNAPSHOT
2.3.4	0.6-SNAPSHOT
2.3.5 – 2.3.11	0.7.0-incubating
2.3.12 – 2.3.13	1.1.1

Iceberg Connector
The latest versions of SeaTunnel usually support relatively new Iceberg versions (such as 1.6.1+).

Common component compatibility reference

For infrastructure components such as Hadoop and Hive, SeaTunnel supports a wide compatibility range, but the following dependencies still need attention:

Component	Compatibility Notes
Hadoop	Supports Hadoop 2.x and 3.x. Object storage connectors such as OSS/OBS usually require Hadoop 2.9+.
Hive	Depends on user-provided JDBC drivers. Ensure the JAR packages under $SEATUNNEL_HOME/lib match the Hive server version.
Spark	When used as an engine, supports Spark 2.4.x and Spark 3.x (depending on the compilation profile).
Flink	When used as an engine, supports Flink 1.14.x – 1.18.x (varies by SeaTunnel version).

5. Service Restart

5.1 Stop the Old Service

Before stopping the service, perform the following checklist:

1. Confirm running tasks: Check whether there are important running jobs.

• Use ./bin/seatunnel.sh -l (Zeta engine) to list running jobs.

  1. Graceful shutdown: Stop jobs using savepoints if you plan to recover them, or wait for batch jobs to finish.

• Stop all SeaTunnel Server nodes (if running in cluster mode).

  1. Metadata backup: If external metadata storage is used (such as JDBC for state storage), back up the relevant database tables.

Stop command:

# Execute on all nodes
sh bin/stop-seatunnel-cluster.sh

5.2 Update Environment Variables

After the upgrade, ensure system environment variables point to the new version directory, otherwise the old version may still be used.

1. Update SEATUNNEL_HOME

If you use separate installation directories (recommended for coexistence of versions), modify environment variables:

vim ~/.bashrc
# or
vim /etc/profile

Update to the new version path:

export SEATUNNEL_HOME=/opt/apache-seatunnel-2.3.x
export PATH=$SEATUNNEL_HOME/bin:$PATH

Apply changes:

source ~/.bashrc
# or
source /etc/profile

Verify:

which seatunnel.sh
echo $SEATUNNEL_HOME

Confirm that the output path is the new version directory.

1. Check Java and engine environment

java -version
echo $JAVA_HOME
echo $SPARK_HOME
echo $FLINK_HOME

Update if necessary:

export JAVA_HOME=/path/to/jdk11

1. Cluster node consistency check

In cluster mode, ensure:

• All nodes point to the same SEATUNNEL_HOME version
• All nodes use the same JAVA_HOME version
• PATH configuration is consistent

Run on each node:

echo $SEATUNNEL_HOME

Avoid a situation where some nodes are upgraded while others still run the old version.

5.3 Start the New Service

# Execute on all nodes
sh bin/start-seatunnel-cluster.sh -d

6. Verification and Rollback

6.1 Verification

Run a simple test job (for example, fake source to console sink) to verify core functionality.

sh bin/seatunnel.sh --config config/v2.batch.config.template -e local

Check logs logs/seatunnel-engine-server.log and logs/seatunnel-engine-client.log to ensure no errors occur.

6.2 Rollback Plan

If the upgrade fails or a serious bug is discovered:

1. Stop the new version service.
2. Restore the old installation directory (from the backup).
3. Start the old version service.
4. Verify that the old service runs normally.

7. Common Issues and Notes

7.1 Cluster Upgrade Strategy

The Zeta engine does not support rolling upgrades.
Because different versions may have incompatible communication protocols or serialization formats, it is strictly forbidden to run a mixed-version cluster.

• Correct approach: stop all old nodes → upgrade all nodes → start all new nodes.

7.2 Environment Variable Check and Fix

If abnormalities occur after the upgrade (such as startup scripts still pointing to the old version), check the following:

1. Whether multiple SEATUNNEL_HOME definitions exist (for example both .bashrc and /etc/profile).
2. Whether the old path still exists in PATH.
3. Whether a symbolic link was used for version switching but not updated.

Recommended command:

echo $PATH | tr ':' '\n' | grep seatunnel

Ensure only the new version path is included.

If using symbolic links to manage versions:

ln -sfn /opt/apache-seatunnel-2.3.x /opt/seatunnel

Ensure all scripts and services point to the current version.

7.3 Common Error Troubleshooting

• ClassNotFoundException / NoClassDefFoundError

  • Check whether third-party JAR files (such as JDBC drivers) from the old version were migrated to the new lib directory.
  • Check whether plugin_config specifies the correct connector name.

• IncompatibleClassChangeError / NoSuchMethodError

  • Usually caused by dependency conflicts. Check whether duplicate JAR files exist in the lib directory (for example both guava-27.jar and guava-30.jar).