DEV Community

Cover image for Stop Losing Background Jobs: Meet Masterly.NightShift for .NET
Ahmad Al-Freihat
Ahmad Al-Freihat

Posted on Edited on

Stop Losing Background Jobs: Meet Masterly.NightShift for .NET

Ever found yourself debugging a background task that mysteriously disappeared after an application restart? Or trying to coordinate multiple dependent jobs while dealing with retries, duplicate execution, and concurrency issues?

There's a better way.

Introducing Masterly.NightShift šŸŒ™

Masterly.NightShift is an open-source background job processing engine for .NET 8+, built on a modernized fork of Hangfire 1.8.25.

It brings durable execution, workflow orchestration, resilience, and observability together under a familiar API.

dotnet add package Masterly.NightShift
Enter fullscreen mode Exit fullscreen mode

The Problem

Background processing often starts with something like this:

// In your controller
[HttpPost("register")]
public IActionResult Register(RegisterRequest request)
{
    var user = CreateUser(request);

    // Fire and forget?
    _ = Task.Run(() => SendWelcomeEmail(user.Id));

    return Ok();
}
Enter fullscreen mode Exit fullscreen mode

Looks simple. But consider what happens when:

  • The application restarts before the task completes.
  • An external API throws an exception.
  • The request's scoped dependencies are disposed.
  • Multiple workers attempt the same operation.
  • You need to coordinate dependent operations.

Task.Run() doesn't provide durable storage, distributed execution coordination, or automatic recovery.

As your requirements grow, you eventually need a proper background processing engine.

The Solution

With NightShift, you can enqueue persistent background jobs using a familiar API:

jobs.Enqueue<EmailService>(
    x => x.SendWelcome(userId));
Enter fullscreen mode Exit fullscreen mode

NightShift persists the job, processes it through a background worker, and manages its execution state.

The job remains available through application restarts, and failures can be handled through retry policies.

No separate Windows Service or mandatory message broker required.

Why Developers Should Check It Out

1. Flexible Job Scheduling

Run jobs immediately, after a delay, on a recurring schedule, or after another job completes.

// Fire-and-forget
jobs.Enqueue<EmailService>(
    x => x.SendWelcome(userId));

// Delayed execution
jobs.Schedule<ReminderService>(
    x => x.SendTrialEnding(accountId),
    "tomorrow at 9am");

// Recurring execution
recurringJobs.AddOrUpdate<ReportService>(
    "daily-report",
    x => x.BuildDaily(),
    Cron.Daily(6));

// Continuation
jobs.ContinueJobWith<EmailService>(
    chargeId,
    x => x.SendReceipt(orderId));
Enter fullscreen mode Exit fullscreen mode

No complicated scheduling infrastructure for common background operations.

2. Workflow Orchestration

Need to execute multiple dependent jobs?

NightShift provides a fluent job-chain API:

JobChain
    .StartWith<OrderService>(
        x => x.Process(orderId))
    .Then<InvoiceService>(
        x => x.Render(orderId))
    .ThenMany(branches => branches
        .Branch<Mailer>(
            x => x.SendInvoice(orderId))
        .Branch<ErpClient>(
            x => x.Post(orderId)))
    .ThenAfterAll<Notifier>(
        x => x.OrderDone(orderId))
    .Enqueue();
Enter fullscreen mode Exit fullscreen mode

This workflow:

  • Processes an order.
  • Generates an invoice.
  • Executes email delivery and ERP synchronization in parallel.
  • Runs the final notification after both branches complete.

No manually maintained dependency graph or custom workflow coordinator.

3. Built-in Resilience

External APIs fail. Servers restart. Distributed workloads experience concurrency conflicts.

NightShift includes mechanisms for handling these situations:

  • Automatic retries with configurable backoff.
  • Circuit breakers for unhealthy dependencies.
  • Duplicate prevention.
  • Distributed concurrency controls.
  • Rate limiting and throttling.
  • Timeouts and execution deadlines.

These are especially useful when building integrations with external APIs or processing large workloads across multiple workers.

4. Entity Framework Core Integration

Ever saved an order successfully but failed to enqueue its background job?

NightShift's EF Core storage integration supports transactional job creation.

db.Orders.Add(order);

db.NightShift().Enqueue<IEmailSender>(
    x => x.SendConfirmation(order.Id));

await db.SaveChangesAsync();
Enter fullscreen mode Exit fullscreen mode

With supported configuration, the order and job commit together.

This helps prevent inconsistencies between your business data and background processing state.

5. Multiple Storage Providers

NightShift supports several storage options:

Storage Support
SQL Server āœ…
Entity Framework Core āœ…
Redis āœ…
MongoDB āœ…
In-memory āœ…

Use infrastructure that already fits your application.

6. Advanced Dashboard and Observability

Background jobs shouldn't be a black box.

NightShift provides a built-in dashboard with:

  • Real-time execution monitoring.
  • Failure insights and clustering.
  • Job lineage and dependency tracking.
  • Worker and queue monitoring.
  • Progress tracking and diagnostics.

It also integrates with OpenTelemetry, Prometheus, health checks, and alerting systems.

7. Modern Performance Improvements

NightShift modernizes important parts of the execution pipeline using:

  • System.Text.Json.
  • Source-generated job invocation.
  • Optimized SQL operations.
  • Bulk job creation.

The project's .NET 10 benchmarks against Hangfire 1.8.25 report:

Benchmark Hangfire NightShift
Direct invocation 17.0 ns 3.5 ns
Serialization allocation 10.97 KB 1.84 KB
Concurrent SQL enqueue 1,523 jobs/s 3,179 jobs/s

That's approximately 4.9x faster direct invocation, 83% less serialization memory, and 2.1x higher concurrent enqueue throughput in the measured workloads.

Actual results depend on your infrastructure and workload. See the repository's benchmarks for the methodology.

Real-World Example: Order Processing

Imagine you're building an e-commerce application.

When an order is created, you need to process it, generate an invoice, notify the customer, and synchronize with an ERP.

Instead of embedding everything inside the HTTP request, define the operations as background jobs.

app.MapPost("/orders/{id:int}/process",
    (int id, IBackgroundJobClient jobs) =>
{
    jobs.Enqueue<OrderProcessor>(
        x => x.Process(id));

    return Results.Accepted();
});
Enter fullscreen mode Exit fullscreen mode

Your API responds immediately, while the worker processes the operation in the background.

For more complex processes, use job chains to express execution dependencies.

Key Features at a Glance

Feature Description
Persistent jobs Survive restarts and deployments
Scheduling Immediate, delayed, recurring, continuation
Workflows Sequential and parallel job chains
Resilience Retries, circuit breakers, duplicate prevention
Distributed controls Throttling, rate limiting, concurrency limits
EF Core integration Transactional job creation
Observability Dashboard, tracing, metrics, health checks
Storage SQL Server, EF Core, Redis, MongoDB, in-memory
Performance Source-generated invocation, optimized serialization

Getting Started

Install the package:

dotnet add package Masterly.NightShift
Enter fullscreen mode Exit fullscreen mode

Configure it in your ASP.NET Core application:

using Masterly.NightShift;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddNightShift(config => config
    .SetDataCompatibilityLevel(
        CompatibilityLevel.Version_180)
    .UseSimpleAssemblyNameTypeSerializer()
    .UseRecommendedSerializerSettings()
    .UseSqlServerStorage(
        builder.Configuration
            .GetConnectionString("NightShift")!));

builder.Services.AddNightShiftServer();

var app = builder.Build();

app.MapNightShiftDashboard();

app.Run();
Enter fullscreen mode Exit fullscreen mode

Configure your SQL Server connection string, then navigate to /nightshift to access the dashboard.

Remember to configure dashboard authorization for production deployments.

When Should You Use It?

NightShift is worth evaluating if you're:

  • Building modern .NET applications.
  • Working with complex background workflows.
  • Integrating with unreliable external APIs.
  • Managing distributed background workloads.
  • Looking for integrated monitoring and resilience.
  • Evaluating alternatives within the Hangfire ecosystem.

NightShift 1.0 is an early release. Test compatibility and production requirements carefully.

It's also an independent fork, not affiliated with or endorsed by Hangfire OÜ.

Conclusion

Background processing shouldn't require you to reinvent retries, orchestration, distributed coordination, and observability.

With Masterly.NightShift, you get:

  • Reliable and persistent background execution.
  • Flexible job scheduling.
  • Fluent workflow orchestration.
  • Built-in resilience and execution controls.
  • Modern monitoring and diagnostics.
  • Optimizations for modern .NET applications.

Your background jobs deserve better than a collection of fire-and-forget tasks.

Give NightShift a try and see how it fits your architecture.


Resources

šŸ”— GitHub: https://github.com/a7mdfre7at/Masterly.NightShift

šŸ“– Documentation: https://a7mdfre7at.github.io/Masterly.NightShift/

šŸ“¦ NuGet: https://www.nuget.org/packages/Masterly.NightShift

If you find the project useful, consider giving it a ⭐ on GitHub!

What's your biggest challenge with background processing in .NET — reliability, workflow orchestration, performance, or monitoring? Drop a comment below!

Top comments (0)