DEV Community

Cover image for If You Can't Explain Why a Background Job Runs, You Don't Control Your System
Oleksandr Viktor
Oleksandr Viktor

Posted on

If You Can't Explain Why a Background Job Runs, You Don't Control Your System

#dotnet #backend #workflow #aspnetcore

Background job systems are everywhere.

Send an email.

Generate a report.

Process a payment.

Synchronize data.

Most applications start simple:

Job
 ↓
Execute
 ↓
Done
Enter fullscreen mode Exit fullscreen mode

Then a few months later:

Job
 ↓
Retry
 ↓
Pipeline
 ↓
Middleware
 ↓
Filters
 ↓
Background magic
 ↓
???
Enter fullscreen mode Exit fullscreen mode

And eventually someone asks:

Why did this job run?

Nobody is quite sure.

Imagine a production incident.

A background job sent thousands of emails.

The logs show success.

The dashboard shows success.

The scheduler shows success.

Yet nobody can answer a simple question:

Why did this job run?

That question became the starting point for my design decisions.

When Execution Becomes Hard to Follow

Modern background processing systems provide powerful abstractions.

The problem is that every abstraction introduces another place where execution flow can hide.

Questions become surprisingly difficult:

  • Why was this job retried?
  • What scheduled the next step?
  • Which component modified execution?
  • Why did this workflow reach its current state?

The system works.

But understanding the system becomes harder over time.

A Different Approach

While working on workflow and job execution systems, I kept coming back to a simple idea:

Every execution step should be explicit.

Instead of hidden orchestration:

Job
 ↓
Framework
 ↓
Magic
 ↓
Result
Enter fullscreen mode Exit fullscreen mode

I prefer:

Job
 ↓
Executor
 ↓
Action
 ↓
ActionResult
 ↓
JobCommand
Enter fullscreen mode Exit fullscreen mode

Every step is visible.

Every transition is intentional.

Every workflow can be traced by reading code.

Workflows Should Be Code

Consider a simple process:

Send Email
 ↓
Write Log
 ↓
Done
Enter fullscreen mode Exit fullscreen mode

In WJb, an action explicitly decides what happens next.

return ActionResults.Next(
    new JobCommand(
        Actions.Log,
        new LogInput
        {
            Message = $"Email sent to {input.To}"
        }));
Enter fullscreen mode Exit fullscreen mode

The workflow is not hidden in configuration.

The workflow is not hidden in a dashboard.

The workflow is not hidden inside middleware.

It is right there in the action.

Traditional Thinking vs Explicit Execution

Traditional approach:

Job
 ↓
Scheduler
 ↓
Retry Logic
 ↓
Filters
 ↓
Pipeline
 ↓
Result
Enter fullscreen mode Exit fullscreen mode

Explicit execution:

Job
 ↓
Action
 ↓
ActionResult
 ↓
JobCommand
 ↓
Next Action
Enter fullscreen mode Exit fullscreen mode

The difference is not performance.

The difference is understanding what happens when something goes wrong.

What Does a Minimal Workflow Look Like? 

send-email → log → done
Enter fullscreen mode Exit fullscreen mode 
await wjb.EnqueueAsync(
    Actions.SendEmail,
    new EmailInput
    {
        To = "user@test.com"
    });

await wjb.ExecuteLoopAsync();
Enter fullscreen mode Exit fullscreen mode

The complete example is small enough to understand in a few minutes.

No special DSL.

No separate workflow engine.

No conventions that require documentation to discover.

Explicit Retries

Retries are part of the job configuration.

await wjb.EnqueueAsync(
    RetryAction.Key,
    new EmptyInput(),
    new JobOptions
    {
        MaxRetries = 3,
        RetryDelay = TimeSpan.FromSeconds(10)
    });
Enter fullscreen mode Exit fullscreen mode

Execution flow remains predictable:

Attempt 1
 ↓
Failed
 ↓
Retry Job Created
 ↓
Attempt 2
 ↓
Completed
Enter fullscreen mode Exit fullscreen mode

Nothing mysterious happens behind the scenes.

The Question I Use During Reviews

When reviewing a background processing system, I ask:

  • Why did this job run?
  • Who scheduled it?
  • Why was it retried?
  • What happens next?

If answering those questions requires opening dashboards, configuration files, middleware registrations and framework internals, the execution model may be more complicated than necessary.

Why I Built WJb

The goal was never to create another scheduler.

The goal was to build a system where you can answer four simple questions:

  • Why did a job start?
  • What did it do?
  • What did it schedule next?
  • Why was it retried?

If those questions are difficult to answer, debugging production systems becomes harder than it should be.

Predictability First, Performance Second

Predictability was the primary goal.

Performance turned out to be a nice side effect.

Recent BenchmarkDotNet measurements on .NET 10 produced:

CreateOnly       64.9 ns
ExecuteOnly       8.9 ns
CreateAndExecute 75.1 ns
Enter fullscreen mode Exit fullscreen mode

Not bad for a lightweight open source project.

Getting Started

Explore the demo project and examples:

👉 https://github.com/UkrGuru/WJb.Demo

Browse all WJb packages:

👉 https://www.nuget.org/packages?q=wjb

Is WJb Right for You?

WJb may be interesting if you want:

  • explicit workflows
  • strongly typed actions
  • predictable execution
  • constructor injection
  • simple orchestration
  • testable business logic

It may not be the right tool if you prefer:

  • convention-driven behavior
  • hidden orchestration
  • execution pipelines that aren't visible in code

Closing Thoughts

Background jobs should not feel mysterious.

A workflow should be understandable by reading the code that defines it.

That's the principle behind WJb:

Action       = Business Logic
ActionResult = Outcome
JobCommand   = Next Step
Executor     = Runner
Store        = Persistence
Enter fullscreen mode Exit fullscreen mode

You should always know:

  • why a job started
  • what it did
  • what it scheduled next
  • why it was retried

No magic.

Just explicit execution.

NuGet Packages:

https://www.nuget.org/packages?q=wjb

Demo Repository:

https://github.com/UkrGuru/WJb.Demo

Top comments (0)