DevLog 20250807: Microservice for Cloud Scheduling (and Running)

Motivation

When working with routine tasks, it's convenient to have some form of automatic scheduling. Windows has Task Scheduler, Linux has cron, and in AWS, you can use Lambda with EventBridge.

Wouldn't it be nice if we could periodically pull data from the web, do some light processing, and then dump the results into a database - maybe update a website, trigger a CI/CD process, or send a few emails?

I haven't used it, but I imagine that's what Zapier is designed for. Azure/GitHub Actions are certainly handy for CI/CD. But what if we just want to quickly prototype something, ship it to a server, and forget about it?

That's the vision, anyway.

The First Attempts

When I first started building services for cloud computing, I combined both "trigger a run" and "schedule repeated tasks" into a single service. That sort of worked - but it didn't feel elegant. Something was off.

Even now, after deciding to completely separate the "runner" and "scheduler" into two distinct microservices, I can't fully articulate why this approach feels better. It just does. Of course, this means maintaining three or four separate programs now (runner, runner service, scheduler, scheduler service) - but each one is responsible for a single task, and that somehow feels more reassuring.

Separation of Concerns & the Single Responsibility Principle

I asked ChatGPT for a rationale that aligned with my intuition, and here's what it gave me:

What you're discovering is a classic case of separation of concerns (and in the microservice world, the "scheduler-worker" or "cron-vs-runner" pattern), underpinned by the Single Responsibility Principle. Here's why splitting "who decides when" from "what actually does the work" just feels better.

1. Single Responsibility & Cognitive Load

• Scheduler service - its sole job is: "At 8 AM on Tuesdays, kick off job X."
• Runner service - its sole job is: "When told to run X, do the fetch/process/dump."

By assigning each service a clear, focused purpose, you reduce logic complexity in any single codebase. You don't need to sift through your fetch logic to understand date calculations, and you don't need to worry about HTTP clients inside your scheduler.

Feels better because each code path is narrow, easy to reason about, and small enough to hold entirely in your head.

2. Loose Coupling & Replaceability

When the scheduler and runner communicate through a simple interface (e.g., message queue, webhook, or HTTP call), either can be swapped out independently:

• Want to switch from cron-style fixed schedules to GitHub webhook events?
• Need to rewrite your worker in another language for better library support?

You only have to reimplement one microservice - not the whole system.

Feels better because the impact of change is localized.

3. Independent Scaling & Reliability

Scheduling frequency and execution workload rarely match 1:1.

• Maybe you schedule jobs every minute, but each run only takes a few seconds.
• Or you only need one scheduler, but dozens of parallel workers to clear a backlog.

Separating the two allows for independent sizing, deployment, and monitoring.

Feels better because you avoid resource contention (e.g., a busy worker delaying the next schedule) and can autoscale each component based on its own metrics.

4. Clear Boundaries for Observability & Error Handling

• Scheduler logs should only show: "Job X was triggered at time T."
• Runner logs should only show: "Processing of job X succeeded/failed."

This allows for more targeted alerting and diagnostics:

• Alert if a scheduled trigger never reaches the runner.
• Alert if the runner fails more than N times, regardless of the schedule.

Feels better because root cause analysis is straightforward - you know where to look first.

5. Pattern Analogues & Ecosystem Support

This approach mirrors patterns built into major platforms:

• AWS CloudWatch Events (scheduler) → Lambda (worker)
• Kubernetes CronJob (scheduler) → Job pods (workers)
• Celery Beat (scheduler) → Celery workers (task executors)

All of them separate the "when" from the "what," and you're starting to feel the same advantages they offer at scale.

In Summary

You're gravitating toward a Scheduler–Worker (or "Cron-as-a-Service" + "Stateless Runner") pattern because it:

1. Reduces cognitive load by narrowing each service's scope
2. Decouples timing logic from business logic
3. Enables independent scaling and more precise reliability controls
4. Improves testability (you can mock the scheduler in runner tests, and vice versa)
5. Aligns with proven cloud-native and messaging patterns

That "reassuring" feeling is the comfort of a clean, modular architecture - you can see exactly what does what, and update or extend each part without unraveling the rest.

The Analogy

Here is the way I understand it: a rough analogy might be batch-processing vs programming - when chaining multiple operations, it's better to use shell scripting than to rely on spawning subprocesses directly in your source code.