What does it look like when an Agent does more than describe a failed data pipeline? This article documents a real, isolated local run in which Astron Agent called dolphin-mcp-pilot, inspected a failed Apache DolphinScheduler process, drilled into the failed task log, submitted one rerun after a human fixed the external dependency, and kept polling until the scheduler reported SUCCESS.
The screenshots are captures from the running software. This is not a production deployment, and it is not a claim of fully autonomous remediation.
What dolphin-mcp-pilot is
dolphin-mcp-pilot is an open-source MCP tool layer for Apache DolphinScheduler. Instead of asking a model to invent shell commands or scrape a UI, an MCP client can discover structured tools for projects, workflow definitions, process instances, task instances, schedules, resources, logs, monitoring, and raw API access.
Version v0.3.0 registers 58 tools in its source. That number describes the source-registered tool surface in the tested version; it is not a claim that every tool was exercised in this run.
The tested chain
The system under test was:
- Astron Agent
v1.1.1 - dolphin-mcp-pilot
v0.3.0 - Apache DolphinScheduler
3.2.2 - a local standalone, isolated test project
- a custom Streamable HTTP MCP endpoint configured in an Astron ReACT decision node
The project also supports stdio, but this recording demonstrates Streamable HTTP. It does not claim an SSE run, production readiness, or end-to-end Token authentication. No credential is shown in the evidence.
1. Connect the MCP endpoint
The Astron ReACT decision node was configured with a custom MCP URL reachable from the Astron runtime. If Astron runs in a container, the endpoint must be addressed from that container's network context; localhost inside a container is not automatically the host machine.
Astron then discovered the 58 tools registered by the tested dolphin-mcp-pilot source. The useful point is not the count alone: the ReACT node can select and invoke a structured operation while preserving the tool name and returned data in the trace.
2. Observe a real FAILURE
The test process instance was deliberately allowed to enter the real DolphinScheduler terminal state FAILURE. This is scheduler evidence, not a recreated status card.
3. Use next_action correctly
The tool response included next_action, which led the Agent to inspect task nodes and the failed-task log. There is an important boundary here: next_action is a static hint generated by code from the current state. It is not a diagnosis produced by the model.
The failed-task log provided the evidence needed to locate the controlled external dependency. The external condition was then fixed by a human. The demo does not claim that the Agent repaired that dependency itself.
4. Rerun once, then keep checking
After the human fix, the Agent made one ds_rerun_from_failure call. DolphinScheduler returned submitted.
submitted is not success. In this trace it only means that DolphinScheduler accepted START_FAILURE_TASK_PROCESS. The Agent continued polling instead of treating the accepted request as the final result. Recovery was claimed only after the real scheduler state reached SUCCESS; the run count changed from 1 to 2.
Compatibility note: the isCache workaround
In the tested pairing of dolphin-mcp-pilot v0.3.0 and DolphinScheduler 3.2.2, task parameters generated by ds_create_workflow and ds_create_dag_workflow omitted the isCache field required by DolphinScheduler 3.2.2. The demo therefore created the workflow definition through ds_raw_post with a complete payload containing isCache.
This is an observed workaround for this version pair. It should not be generalized to every DolphinScheduler version, and the demo does not imply that either builder succeeded in this environment.
What the run demonstrates
The evidence supports five concrete capabilities:
- Pipeline inspection from a natural-language request through a real MCP tool call.
- Fault localization by moving from a failed process to its task nodes.
- Log drill-down through structured Agent-callable operations.
- Controlled recovery with a human fix before a single rerun-from-failure request.
- Result verification by polling to a real terminal scheduler state.
The same pattern can support supervised data-engineering and ETL operations: routine inspection, failure triage, log collection, controlled recovery, and post-action verification. Moving it into production would still require deployment-specific authentication, authorization, network controls, approval gates, idempotency, audit logging, retry policy, and operational testing.
Watch the bilingual demos
- Chinese short: Astron Agent ReACT — from FAILURE to real SUCCESS
- English short: Astron Agent ReACT — from FAILURE to real SUCCESS
Top comments (0)