My background job failed silently for days. Here's how I catch it now.

Krasimir Petkov — Sat, 02 May 2026 17:42:41 +0000

I found out one of my background jobs had stopped running only after the data looked wrong the next day.

There was no dramatic crash. No big incident. The job just quietly failed, and I only noticed because something downstream looked stale.

That is the annoying part about cron jobs and scheduled scripts. Most of the time they run in the background, write some logs, and nobody thinks about them until something is missing.

I have a few jobs like this:

data updates
cleanup scripts
small imports
external API calls
recurring background tasks

None of them are very exciting. But when one of them does not run, or starts and never finishes, it can create a surprisingly annoying problem.

That is the kind of failure I wanted to make more visible.

I also built a small V1 of this idea here:

https://missedrun.com

There is also a self-hosted version for people who prefer to run this kind of monitoring on their own infrastructure:

missedrun / missedrun-selfhosted

Self-hosted cron job monitoring with ping URLs and email alerts.

MissedRun Self-hosted

Self-hosted cron and scheduled job monitoring for detecting silent failures.

MissedRun monitors recurring jobs such as cron scripts, backups, imports, ETL pipelines, billing syncs, cleanup tasks, and scheduled reports.

It works by giving each monitor a unique ping URL. Your job calls that URL when it runs, starts, finishes successfully, or fails. If the job does not check in within the expected interval plus grace period, MissedRun marks it as missing and can send an alert.

Hosted version: https://missedrun.com
Self-hosted version: https://github.com/missedrun/missedrun-selfhosted
License: AGPL-3.0

What problem does it solve?

Some production failures are not loud.

A job can stop running without throwing an exception. For example:

cron did not run
the server was down
a Docker container stopped
credentials expired before the job reached your alerting code
a backup script never started
an import stopped updating data
a scheduled report was not generated
a background worker silently stopped

…

View on GitHub

This is not a big launch. I am mostly trying to understand if this is a real enough problem for other developers who run cron jobs, ETL jobs, backups, imports, cleanup scripts, or other scheduled tasks.

The problem

Cron jobs are easy to forget about.

They usually do not have a UI. They run somewhere on a server, maybe write logs, and then disappear into the background.

A job can fail because:

an API token expired
an environment variable is missing
a database connection failed
the server restarted
the script crashed
the job started but never finished
the cron entry was changed or removed

Logs are useful, but only if you go and check them.

In practice, I usually only check logs after I already suspect something is broken.

For recurring jobs, I often want a much simpler answer:

did it start?
did it finish?
did it fail?
did it miss the expected time?

The ping approach

One simple way to monitor this is to make the job report its own status.

The basic pattern is:

send a start ping when the job begins
send a success ping when it finishes
send a failure ping if it crashes
mark it as late or missed if the expected ping does not arrive

It is not a complicated idea, but I have found it very useful in practice.

Instead of checking logs manually, the job tells you whether it is still alive.

For example:

if the start ping arrives, the job is running
if the success ping arrives, the job finished
if the fail ping arrives, the job crashed
if nothing arrives when expected, the job is late or missed

That last case is the important one for me.

A lot of failures are not loud. The job does not always send an error. Sometimes it just does not run.

Bash example

Here is a simple shell wrapper.

This uses placeholder URLs. In a real setup, these would be the ping URLs generated by your monitoring tool.

#!/bin/bash

START_URL="https://example.com/ping/YOUR_TOKEN/start"
SUCCESS_URL="https://example.com/ping/YOUR_TOKEN"
FAIL_URL="https://example.com/ping/YOUR_TOKEN/fail"

curl -fsS -X POST --max-time 5 "$START_URL" >/dev/null || true

your-real-command-here

EXIT_CODE=$?

if [ $EXIT_CODE -eq 0 ]; then
  curl -fsS -X POST --max-time 5 "$SUCCESS_URL" >/dev/null || true
else
  curl -fsS -X POST --max-time 5 "$FAIL_URL" >/dev/null || true
fi

exit $EXIT_CODE

Useful links

I also wrote a few short pages around the specific cases I keep running into:

Cron job monitoring guide: https://missedrun.com/cron-job-monitoring
Detect missed cron jobs: https://missedrun.com/missed-cron-job
Self-hosted cron monitoring: https://missedrun.com/self-hosted-cron-monitoring

Building a Real-Time Stock Scanner with Redis, WebSockets, and Postgres

Krasimir Petkov — Thu, 22 Jan 2026 17:32:33 +0000

Background

This project started as an internal tool to understand unusual volume and intraday activity across thousands of US stocks.

I recently built a real-time stock scanner that ingests thousands of symbols, processes live updates, and serves them to users with minimal delay. This post is a short breakdown of the architecture decisions that worked (and the ones that didn’t).

The Core Problem

Market data is:

high-frequency
bursty
time-sensitive

Writing every update directly to a relational database quickly becomes expensive and unnecessary — especially when most of the data is only relevant for seconds or minutes.

Architecture Overview

The system is split into three layers:

Ingestion — pulling snapshot and streaming data from external APIs
Processing — normalizing, filtering, and calculating derived metrics
Distribution — pushing live updates to the frontend

At a high level:

Redis handles hot, ephemeral data
Postgres stores validated, historical data
WebSockets deliver updates to clients in real time

Why Redis for Live Data

Redis turned out to be a much better fit than Postgres for intraday updates:

In-memory speed
Natural TTL support for expiring symbols
Pub/Sub for fan-out
Zero WAL / IO pressure

Postgres is still excellent — just not for data that changes every few seconds.

WebSockets for Distribution

Instead of polling:

active symbols are tracked dynamically
only “in-play” tickers receive updates
clients subscribe to exactly what they need

This keeps both bandwidth and server load under control.

What I’d Do Differently Next Time

Separate “hot” and “cold” data earlier
Treat intraday data as disposable by default
Avoid over-modeling early — schemas can come later

Live Demo

The project is live here:

https://zipston.com

It’s still evolving, but the core real-time pipeline is already doing the heavy lifting.

Closing Thoughts

Real-time systems are less about frameworks and more about respecting the nature of your data.

If your data expires quickly, your architecture should too.
Happy to answer questions or discuss real-time data tradeoffs.

DEV Community: Krasimir Petkov