I'm contributing to the Debezium Platform as part of Google Summer of Code 2026, and I wanted to share what I've been building for the past few weeks: an automated host provisioning system that takes a bare-metal server from "empty Ubuntu box" to "ready to run CDC pipelines" with zero user intervention.
The Problem
The Debezium Platform already works great on Kubernetes. You define a pipeline, the Operator deploys it, done. But not everyone runs Kubernetes. Plenty of teams have VMs or bare-metal servers sitting in a datacenter, and they want to run Debezium there too.
The challenge: before you can deploy a pipeline to a server, that server needs Docker installed, the Host Agent running, and the Debezium Server image cached. Doing this manually on every machine is tedious and error-prone. What if we could automate the entire thing?
The Architecture
The design (documented in DDD-41) introduces a simple contract: the sysadmin adds a host to their ~/.ssh/config file, and the platform takes it from there.
Here's the flow:
User edits ~/.ssh/config (adds a new host)
↓
SshConfigWatcherService detects the change
↓
HostProvisioningService generates a token, submits work to thread pool
↓
AnsibleHostProvisioner builds the command and launches ansible-playbook
↓
Ansible connects via SSH, runs the playbook on the remote host
↓
Host marked READY in the database
Three key components make this work:
SshConfigWatcherService monitors the SSH config file using Java NIO's WatchService. When the file changes, it parses the new entries, diffs them against the database, and triggers provisioning for any new hosts. A scheduled fallback reconciliation runs periodically to catch events that WatchService might miss (especially relevant on NFS mounts and Kubernetes ConfigMap volumes).
HostProvisioningService is a thin orchestrator. It manages status transitions (PENDING → PROVISIONING → READY or FAILED), generates authentication tokens for the Host Agent, and delegates the actual work to a thread pool so the main application threads never block.
AnsibleHostProvisioner is where the real work happens. It builds an ansible-playbook command, launches it as an OS process, and drains stdout/stderr line-by-line to prevent pipe buffer deadlocks.
The Playbook
The Ansible playbook (host-setup.yml) is a single, idempotent recipe that transforms a fresh server into a Debezium-ready host. Here's what it does:
- Bootstrap Python on the target (Ansible needs Python, but minimal server images often don't have it)
- Gather facts about the OS (Debian vs RedHat, x86 vs ARM)
- Install Docker from the official Docker repository (not the distro's outdated version)
- Deploy the Host Agent as a systemd service with the authentication token baked in
- Pre-pull the Debezium Server image (~500MB) so the first pipeline starts instantly
The playbook supports both Debian/Ubuntu and RHEL/CentOS, and it's safe to re-run. On a second execution, most tasks report ok instead of changed because everything is already in place.
2026-07-31 15:28:34,361 INFO [io.debezium.platform.domain.HostStatusService] (vert.x-worker-thread-1) Created pending host: test-host
2026-07-31 15:28:34,422 INFO [io.debezium.util.Threads] (vert.x-worker-thread-1) Requested thread factory for component HostProvisioningService, id = conductor named = provisioner
2026-07-31 15:28:34,424 INFO [io.debezium.util.Threads] (vert.x-worker-thread-1) Creating thread debezium-hostprovisioningservice-conductor-provisioner-0
2026-07-31 15:28:34,426 INFO [io.debezium.platform.environment.host.discovery.SshConfigWatcherService] (vert.x-worker-thread-1) Reconciliation complete: 1 added, 0 removed, 0 updated
2026-07-31 15:28:34,581 INFO [io.debezium.platform.environment.host.provisioning.HostProvisioningService] (debezium-hostprovisioningservice-conductor-provisioner-0) Starting provisioning for host test-host
2026-07-31 15:28:34,582 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (debezium-hostprovisioningservice-conductor-provisioner-0) Executing Ansible provisioning command for host test-host
2026-07-31 15:28:35,107 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | PLAY [Provision host for Debezium pipeline deployment] *************************
2026-07-31 15:28:35,107 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Bootstrap Python on target host] *****************************************
2026-07-31 15:28:38,020 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Gather host facts] *******************************************************
2026-07-31 15:28:40,004 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Fail on unsupported OS family] *******************************************
2026-07-31 15:28:40,021 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Install prerequisite packages for Docker repo] ***************************
2026-07-31 15:28:44,461 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Create keyrings directory] ***********************************************
2026-07-31 15:28:44,794 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Download Docker GPG key] *************************************************
2026-07-31 15:28:45,481 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Add Docker APT repository] ***********************************************
2026-07-31 15:28:51,390 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Add Docker CE repository] ************************************************
2026-07-31 15:28:51,408 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Remove podman/runc conflicts] ********************************************
2026-07-31 15:28:51,503 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Install Docker Engine] ***************************************************
2026-07-31 15:30:03,860 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Enable and start Docker service] *****************************************
2026-07-31 15:30:04,432 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Add SSH user to docker group] ********************************************
2026-07-31 15:30:04,861 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Create Host Agent directory] *********************************************
2026-07-31 15:30:05,086 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Deploy Host Agent systemd unit file] *************************************
2026-07-31 15:30:05,622 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Enable and start Host Agent service] *************************************
2026-07-31 15:30:06,329 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Install Python requests library for community.docker modules] ************
2026-07-31 15:30:07,320 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | TASK [Pre-pull Debezium Server Docker image] ***********************************
2026-07-31 15:31:12,230 INFO [io.debezium.connector.common.BaseSourceTask] (pool-13-thread-1) 4 records sent during previous 00:05:00.967, last recorded offset of {server=conductor} partition is {lsn_proc=28330616, lsn_events_processed=1, messageType=UPDATE, lsn_commit=27874968, lsn=28330616, txId=761, ts_usec=1785492071943362}
2026-07-31 15:36:12,753 INFO [io.debezium.connector.common.BaseSourceTask] (pool-13-thread-1) 2 records sent during previous 00:05:00.523, last recorded offset of {server=conductor} partition is {lsn_proc=28331096, lsn_events_processed=1, messageType=UPDATE, lsn_commit=28330752, lsn=28331096, txId=762, ts_usec=1785492372295412}
2026-07-31 15:39:16,168 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | RUNNING HANDLER [Restart Host Agent] *******************************************
2026-07-31 15:39:17,391 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | PLAY RECAP *********************************************************************
2026-07-31 15:39:17,392 INFO [io.debezium.platform.environment.host.provisioning.AnsibleHostProvisioner] (ansible-output-drainer) ansible | test-host : ok=15 changed=9 unreachable=0 failed=0 skipped=3 rescued=0 ignored=0
2026-07-31 15:39:19,011 INFO [io.debezium.platform.environment.host.provisioning.HostProvisioningService] (debezium-hostprovisioningservice-conductor-provisioner-0) Provisioning completed successfully for host test-host in 10m 43s
The Interesting Problems
Real-time Log Streaming
The Ansible process runs for 10+ minutes (most of that is downloading Docker packages and pulling the Debezium image). The original implementation buffered all output and logged it after the process finished. That meant operators were staring at a silent terminal for 11 minutes with no idea what was happening.
I replaced this with a dedicated drainer thread that reads output line-by-line and logs it in real time. But raw Ansible output is incredibly noisy: deprecation warnings, timestamps, empty lines, per-task ok: [host] results. About 100 lines for a single run, most of which are noise.
So I built a filter. Only lines that convey meaningful progress (task headers, play recap, fatal errors) are logged at INFO. Everything else goes to DEBUG. The result: operators see about 15 clean lines instead of 100, and they can tell exactly which step is running at any moment.
Here's what the clean output looks like:
INFO Starting provisioning for host test-host
INFO ansible | PLAY [Provision host for Debezium pipeline deployment]
INFO ansible | TASK [Install Docker Engine]
INFO ansible | TASK [Pre-pull Debezium Server Docker image]
INFO ansible | PLAY RECAP
INFO ansible | test-host : ok=15 changed=9 unreachable=0 failed=0
INFO Provisioning completed successfully for host test-host in 11m 3s
The macOS WatchService Trap
During local development, I noticed a 5-minute delay between editing the SSH config and the platform detecting the change. Turns out, Java's WatchService on macOS uses a PollingWatchService (there's an open JDK bug: JDK-8293067) instead of native filesystem events. On Linux, it uses inotify and detection is instant.
This validated our design decision to include a scheduled fallback reconciliation. On Linux in production, the WatchService handles everything in real time. On macOS during development, the configurable fallback (platform.host.reconciliation-interval) fills the gap.
Token Redaction
The Ansible command includes an authentication token (--extra-vars "agent_token=<uuid>"). If the playbook fails, we capture the full output for debugging. But we can't store raw tokens in log files or database reports. The AnsibleOutputDrainer automatically redacts any occurrence of the token from captured output before returning the result.
What I Learned
Working on this feature taught me things I wouldn't have learned from tutorials:
Process I/O in Java is subtle. If you don't drain stdout/stderr continuously, the OS pipe buffer (64KB) fills up and the child process deadlocks. This isn't something you'd discover in a unit test because the output is too small.
Idempotent playbooks require real thought. Every task needs a state: present or state: started instead of "install" or "run". The playbook must describe the desired end state, not a sequence of commands.
Log levels matter more than you think. The difference between a useful console and an overwhelming one is just a few if statements. But deciding what's "useful" requires understanding what the operator actually needs to know at any given moment.
Code review makes you better. My mentor Mario pushed back on several things that seemed fine to me at first, like hardcoded intervals, inline lambdas, and committed application.properties files. Every piece of feedback had a clear rationale, and the code is genuinely better for it.
Community Demo
I presented the full provisioning flow at the Debezium community meeting on August 4th. The demo walked through the end-to-end journey: adding an SSH config entry, watching the logs stream in real time as Ansible set up Docker and the Host Agent, and seeing the host reach READY status.
Current Status
The PR (#466) is in review. Next up: the deprovisioning flow (host removal) and wiring the Host Agent to actually receive and execute pipeline deployments.
If you're interested in CDC or want to contribute to Debezium, the community is incredibly welcoming. Everything happens in the open: design docs, PRs, weekly meetings.
THANK YOU !!
Top comments (0)