DEV Community

Chen Debra
Chen Debra

Posted on

Apache DolphinScheduler June Community Update: Strengthening Stability Across the Entire Workflow Operations Lifecycle

#programming #apachedolphinscheduler #opensource #datascience

Hi, Community!

Our June community report is here! Instead of focusing on major feature releases, the community spent the month strengthening the platform's foundation and production readiness.

Throughout June, contributors concentrated on improving operational stability by finalizing the 3.4.2 maintenance release, enhancing task failover capabilities, fixing high-concurrency edge cases, introducing sensitive data masking for logs, and addressing numerous production issues across deployment, storage, CI, and engineering workflows.

In this report, we'll walk you through the most important production-ready improvements delivered in June, provide upgrade recommendations for production environments, and recognize our top community contributors. Let's dive in! ✨

Project Highlights

Reporting Period: June 1, 2026 – July 1, 2026

  • Merged Pull Requests: 22
  • Contributors: 9
  • Code Changes: +630 / -473
  • Major Components: Master, API, Storage, Helm, UI, Documentation & Project Governance

June was primarily a maintenance and platform hardening release cycle. Alongside the Apache DolphinScheduler 3.4.2 release and follow-up fixes, the community delivered meaningful improvements in concurrency safety, log security, Helm deployment, S3 storage handling, and documentation quality.

This update is especially valuable for:

  • End users and platform owners who want to quickly understand whether June introduced important stability or usability improvements.
  • Platform engineers and DevOps teams who should pay close attention to failover behavior, Helm deployment, S3 storage improvements, and logging security.
  • Contributors and developers interested in seeing how the community continues investing in long-term platform reliability and technical governance.

Six Improvements That Matter Most to Users

1. Apache DolphinScheduler 3.4.2 Is Fully Released and Maintained

June included both the official 3.4.2 release and follow-up documentation corrections, representing a typical maintenance release cycle focused on polishing production quality.

Related PRs: #18317, #18319

2. Safer Worker Logs with Password Masking

Worker logs now prevent plaintext passwords from being written to log files.

This improvement is particularly important in enterprise and multi-tenant environments, where credential exposure through logs can become a far greater security risk than ordinary software bugs.

Related PR: #18333

3. Improved Helm Deployment Experience

Several deployment-related improvements landed in June, including fixes for duplicate ConfigMap labels and an updated MySQL Helm chart version.

These changes directly improve deployment reliability and long-term maintainability for Kubernetes users.

Related PRs: #18341, #18336

4. Better Concurrency Safety

A key stability improvement replaces HashMap with ConcurrentHashMap, eliminating potential ConcurrentModificationException issues under high-concurrency workloads.

Although these failures were relatively rare, they could have significant production impact when they occurred.

Related PR: #18331

5. Smarter Task Failover

A new failover enhancement allows Apache DolphinScheduler to automatically terminate external applications when task failover occurs.

This is particularly valuable for Yarn and Kubernetes deployments, preventing orphaned applications from continuing to consume cluster resources after failover.

Related PR: #18353

6. S3 Resource Listings No Longer Stop at 1,000 Objects

Users relying on Amazon S3-compatible storage will no longer encounter incomplete resource listings once object counts exceed 1,000.

This fixes a long-standing usability issue that directly affected storage visibility and operational correctness.

Related PR: #18381

Upgrade Recommendations

If you're running Apache DolphinScheduler in production, we recommend validating this release from two perspectives: maintenance updates and platform stability improvements.

Priority validation areas include:

  • Log masking to ensure sensitive credentials are never persisted.
  • High-concurrency workload stability.
  • S3 resource visibility.
  • Helm deployment and upgrade workflows.

We also recommend verifying release documentation and upgrade guides, especially if your organization maintains internal deployment documentation or embedded platform references.

Before incorporating June's updates, we recommend keeping your development environment synchronized using the following commands:

git fetch origin dev
git checkout dev
git pull --rebase
Enter fullscreen mode Exit fullscreen mode

June Contributors

Based on Git author names, nine community members contributed code during June:

  • eye-gu
  • Jarek Potiuk
  • luxiaolong
  • njnu-seafish
  • suyc
  • Victor Laborie
  • Wenjun Ruan
  • xiangzihao
  • Yanjun Qiu

Technical Deep Dive

1. Concurrency Safety: Fixing the Root Cause Instead of the Symptoms

Representative PR: #18331

The UserGroupInformationFactory cache has been upgraded from HashMap to ConcurrentHashMap, accompanied by dedicated concurrency test cases.

The real challenge with concurrency issues isn't that they happen—it's that they occur only under specific high-load conditions and are extremely difficult to reproduce.

When Hive authentication and Kerberos renewal threads accessed the same cache simultaneously, a standard HashMap could trigger ConcurrentModificationException during concurrent iteration and modification.

Instead of applying a temporary synchronization workaround, the community addressed the underlying design by replacing the shared data structure with a thread-safe implementation and introducing dedicated tests that simulate concurrent forEach and logout/login operations.

This fundamentally eliminates the issue rather than masking its symptoms.

For production environments using Hive, Kerberos authentication, or long-lived sessions, this significantly reduces intermittent failures that disappear after retries but remain difficult to diagnose.

Recommended validation focuses on Hive workloads, Kerberos renewal scenarios, and high-concurrency datasource access.

2. Smarter Failover That Cleans Up External Applications

Representative PR: #18353

June introduces a new configuration option, kill-application-when-task-failover, integrated into MasterConfig, task execution logic, and multiple deployment configurations.

Historically, failover primarily ensured scheduling continuity.

However, for external execution engines such as Yarn and Kubernetes, the larger operational risk often comes from orphaned applications that continue consuming resources even after the scheduler has recovered.

This enhancement expands failover beyond scheduler metadata management to include cleanup of execution environments.

The implementation spans core Master logic as well as application.yaml, Docker Compose configurations, Kubernetes Helm values, and documentation, demonstrating that this is a production-ready capability rather than an experimental feature.

For organizations running Yarn or Kubernetes clusters, this provides a unified mechanism for preventing resource leakage after failover events.

Recommended validation includes Master failover testing, interrupted task execution, and verification that legitimate applications remain untouched while orphaned applications are correctly terminated.

3. S3 Pagination Support That Scales Beyond Development Environments

Representative PR: #18381

S3StorageOperator.listStorageEntity() has been enhanced to iterate through paginated results using ContinuationToken instead of relying on a single ListObjectsV2 request.

This addresses a classic production issue.

Development environments rarely exceed 1,000 objects, making the limitation difficult to detect. Once production storage grows beyond that threshold, however, users encounter missing resources even though the underlying files still exist.

Rather than implementing a workaround at the application layer, the community fixed pagination directly within the storage plugin.

Listing now continues until isTruncated() returns false, ensuring every object is returned.

As a result, resource centers, file management interfaces, and directory browsers all benefit automatically.

For organizations using large S3 buckets or treating DolphinScheduler as a centralized resource management platform, this restores correctness—not just convenience.

Recommended validation includes directories containing more than 1,000 objects, mixed folder structures, and confirmation that UI and API results match the actual contents of the storage bucket.

4. Small Change, Major Security Improvement: Log Desensitization

Representative PR: #18333

From a code perspective, this pull request modifies only a small portion of PasswordUtils.

Its impact, however, is far greater than the size of the code change suggests.

Security improvements should never be measured by the number of modified lines.

Plaintext credentials appearing in logs don't usually cause immediate system failures. Instead, they silently propagate through centralized logging platforms, troubleshooting systems, backups, and collaborative workflows, dramatically increasing the attack surface.

This update reflects the community's continued investment in secure-by-default behavior.

Many real-world security incidents originate not from broken authentication systems, but from sensitive information accidentally exposed through ordinary application logs.

For organizations already aggregating Worker logs into ELK, Alibaba Cloud SLS, Amazon CloudWatch, or similar platforms, this enhancement provides even greater value because centralized logging significantly amplifies the potential impact of leaked credentials.

When upgrading, verify that sensitive information is properly masked while ensuring logs still retain sufficient detail for effective troubleshooting.

Development Focus

Looking across all 22 merged pull requests, June represents a classic platform hardening cycle focused on resolving numerous high-frequency production issues that affect reliability over time.

Key investment areas included:

  • Release Management & Project Governance: #18310, #18317, #18319, #18363, #18374, #18375
  • Runtime Stability: #18331, #18347, #18351, #18352, #18353
  • Cloud-Native Deployment: #18336, #18341, #18355, #18360
  • Security & Compliance: #18310, #18333, #18363
  • Storage & Ecosystem: #18381
  • Documentation & User Experience: #18321, #18323, #18327, #18308

If we had to summarize Apache DolphinScheduler's June development in a single sentence, it would be this:

June wasn't about shipping major new features—it was about making Apache DolphinScheduler more reliable, more secure, and better prepared for production by strengthening the operational details that matter most.

Appendix A: Merged Pull Requests in June 2026

  • 2026-06-02 #18310 Add AGENTS.md + SECURITY.md to make the security model discoverable
  • 2026-06-04 #18317 Release 3.4.2
  • 2026-06-04 #18319 Hotfix 3.4.2 doc error
  • 2026-06-05 #18321 Update the parameter priority explanation in the docs
  • 2026-06-05 #18323 Add a note to the parameter priority documentation specifying and fix some issues
  • 2026-06-07 #18316 Remove dead code after t_ds_relation_user_alertgroup table dropped
  • 2026-06-10 #18331 Replace HashMap with ConcurrentHashMap in UserGroupInformationFactory
  • 2026-06-12 #18341 Fix duplicate app.kubernetes.io/name label on ConfigMap
  • 2026-06-12 #18333 Remove plaintext passwords from the logs
  • 2026-06-12 #18336 Update mysql helm chart version
  • 2026-06-15 #18347 countTaskInstanceStateByProjectCodes uses submit_time filtering
  • 2026-06-16 #18327 Fix and enrich the documentation for parameter priority
  • 2026-06-16 #18355 Change the description field to optional
  • 2026-06-17 #18353 Add kill-application-when-task-failover logic
  • 2026-06-17 #18352 Fix rerun workflow instance should follow the specified workerGroup
  • 2026-06-18 #18360 Change the description field to optional in k8s config
  • 2026-06-22 #18351 Fix forceTaskSuccess cannot reset the last unsuccessful workflow instance to success
  • 2026-06-22 #18363 Fix sonar token leak
  • 2026-06-23 #18374 Remove sonar check
  • 2026-06-23 #18375 Fix auto labeler error
  • 2026-06-23 #18308 Frontend correctly uses the preferred values of the associated project for task creating and workflow scheduling
  • 2026-06-26 #18381 Fix list resources returned only 1000 records in s3 storage type

Appendix B: References

Top comments (0)