DEV Community: Stephen Souza

Beyond Uptime: The Complete Monitoring Stack for SaaS Builders

Stephen Souza — Thu, 07 May 2026 04:36:41 +0000

Your uptime monitor says green.

Your server is responding. CPU is normal. No errors in the logs.

But signups stopped 4 hours ago. Nobody noticed.

That's the gap most monitoring stacks have and it's the gap that costs the most.

This is the monitoring stack we run at NotiLens, built for SaaS teams who don't have a dedicated DevOps engineer watching dashboards all day.

The problem with uptime-only monitoring

Traditional monitoring answers one question: is the server on?

What it doesn't answer:

Are users actually signing up?
Are payments completing — not just initiating?
Are cron jobs processing records — not just running?
Are AI agents producing output — not just executing?

These are business-layer failures. Infrastructure monitoring completely misses them.

Here's how to cover both layers.

Layer 1: Revenue monitoring

Stripe webhooks

Stripe webhook failures are the silent killer most SaaS builders don't monitor. Your endpoint can return 200s while silently failing to process events — subscriptions go stale, payment failures go unhandled, refunds queue up.

NotiLens monitors Stripe from two angles simultaneously.

Signal 1 — Stripe sends directly to NotiLens:
Configure a NotiLens webhook endpoint in your Stripe dashboard alongside your existing endpoint. NotiLens receives the raw event.

Signal 2 — Your backend confirms processing:

app.post('/webhooks/stripe', async (req, res) => {
  const event = stripe.webhooks.constructEvent(
    req.body,
    req.headers['stripe-signature'],
    process.env.STRIPE_WEBHOOK_SECRET
  );

  // Your existing processing logic
  await handleStripeEvent(event);

  // Confirm to NotiLens that processing completed
  await notilens.track("stripe.webhook.processed", {
    type: event.type,
    customerId: event.data.object.customer
  });

  res.json({ received: true });
});

What ML detects:

Stripe sent the webhook ✓ but your backend never confirmed processing ✗ → broken flow alert
Both signals arrived but volume dropped below normal baseline → silence alert
Sudden spike in webhook volume → anomaly alert

The gap between Signal 1 and Signal 2 is where most payment failures hide.

→ Stripe webhook monitoring
→ Stripe payment failure alerts

Shopify orders

Configure Shopify to send webhook events directly to NotiLens:

Go to your Shopify Admin → Settings → Notifications
Scroll to Webhooks → click Create webhook
Select event: Order creation and Order payment
Paste your NotiLens Shopify webhook URL
Set format to JSON → Save NotiLens watches incoming order volume against your baseline. If orders go abnormally quiet for your time of day — silence alert fires. No manual threshold needed.

→ Shopify order monitoring
→ Shopify silent order drop alerts

Layer 2: Silence monitoring

This is the most important layer — and the one nobody talks about.

Silence monitoring answers: is anything actually happening?

Your server can be perfectly healthy while:

No new users have signed up in 6 hours
No new orders have come in since midnight
A background job ran but processed zero records
An API is responding but returning empty results

None of these trigger a server alert. All of them are serious.

// Track every signup
await notilens.track("user.signup.completed", {
  userId: user.id,
  plan: user.plan
});

// Track every activated user
await notilens.track("user.activated", {
  userId: user.id
});

NotiLens learns your baseline — how many signups per hour is normal at 2am on a Tuesday — and alerts you when it drops significantly below that. No manual threshold needed.

You can also detect broken flows — user.signup.completed fired but user.activated never followed within 30 minutes:

Layer 3: Infrastructure basics

Keep this minimal. What you actually need:

Server up/down — server downtime alerts
Server silence — server silence monitoring for when your server stops reporting entirely
API error rate spikes — API error rate monitoring

What you probably don't need yet: APM dashboards, distributed tracing, custom metrics pipelines.

Layer 4: Cron jobs and scheduled tasks

The problem isn't when a cron job crashes. It's when it runs successfully but does nothing.

Exit code 0. Zero records processed. No alert.

Fix: heartbeat monitoring. Your job sends a ping on successful completion. If the ping doesn't arrive in the expected window — alert fires.

// At the end of your cron job
const result = await processBillingRecords();

await notilens.track("billing.sync.job", {
  recordsProcessed: result.count,
  duration: result.durationMs
});

NotiLens ML detects two anomalies beyond just "did it run?":

recordsProcessed consistently 0 — job ran but did nothing
durationMs spikes above normal baseline — job is taking significantly longer than usual, often the first sign of a database or dependency issue before it becomes an outage

→ Cron job failure monitoring

Three jobs to instrument first:

Billing sync
Email delivery
Data cleanup / reporting

Layer 5: Developer activity

Use the official NotiLens GitHub Action — no curl needed:

# .github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy to production
        run: ./deploy.sh

      - name: Notify deploy success
        if: success()
        uses: notilens/notify-action@v1
        with:
          token:    ${{ secrets.NOTILENS_TOKEN }}
          secret:   ${{ secrets.NOTILENS_SECRET }}
          event:    task.completed
          message:  "Deployed to production — ${{ github.ref_name }}"
          tags:     deploy,production
          open_url: https://myapp.com

      - name: Notify deploy failure
        if: failure()
        uses: notilens/notify-action@v1
        with:
          token:   ${{ secrets.NOTILENS_TOKEN }}
          secret:  ${{ secrets.NOTILENS_SECRET }}
          event:   task.failed
          message: "Production deployment failed — ${{github.ref_name }}"

The action automatically includes repo, branch, commit, actor, and a direct link to the workflow run — no extra config needed.

→ GitHub CI/CD alerts

Know immediately when a deployment breaks. Don't find out because something stopped working in production.

Layer 6: AI agents and automations

AI agents fail in ways traditional monitoring completely misses:

Silent no-output — runs, completes, exits 0, produces nothing.
Infinite loops — keeps retrying the same step, token costs climb silently.
Stuck tool calls — waiting for a response that never comes.

// Track agent lifecycle
await nl.start('Agent run started', { task: 'report-agent' });

// Track token usage — ML detects anomalous spikes (loops)
await nl.metric({ tokens: response.usage.total_tokens }, { task: 'report-agent' });

// On completion
await nl.complete('Agent completed', { task: 'report-agent' });

// On loop/timeout detection
await nl.timeout('Agent exceeded expected duration', { task: 'report-agent' });

NotiLens detects when token usage spikes above your normal baseline — catches infinite loops before your API bill does.

→ AI agent monitoring

For no-code automation platforms:
→ Zapier workflow failure alerts
→ n8n automation monitoring
→ Make.com automation monitoring

The setup order

Don't try to instrument everything at once.

Week 1 — Revenue first:

Stripe webhook tracking
Payment failure alerts
Shopify order silence (if applicable)

Week 2 — Business health:

Signup silence alert
Server up/down
One critical cron job heartbeat

Week 3 — Operations:

API error rate
GitHub CI/CD failures
Second cron job

Week 4+ — AI and automation:

Agent monitoring
Zapier/n8n/Make workflow monitoring

Start with what touches revenue. Work outward from there.

The full SDK install

npm install @notilens/notilens

import { NotiLens } from '@notilens/notilens';

const nl = NotiLens.init('my-app', { token: 'YOUR_TOKEN', secret: 'YOUR_SECRET' });


// Track a business event
await nl.track('event.name', 'Event description', { meta: { ...metadata } });

// Task lifecycle
await nl.start('Job started', { task: 'job-name' });
await nl.complete('Job done', { task: 'job-name' });
await nl.fail('Job failed', { task: 'job-name' });

// Metrics
await nl.metric({ records: 1500, durationMs: 3200 }, { task: 'job-name' });

Full docs at notilens.com/doc

SDK support

NotiLens has official SDKs for most stacks — no HTTP wiring needed:

# Node.js
npm install @notilens/notilens

# Python
pip install notilens

# PHP
composer require notilens/notilens

# Go
go get github.com/notilens/sdk-go

# Rust
cargo add notilens

# Ruby
gem install notilens

Java and Kotlin available via Maven and Gradle. Shell/CLI also supported — useful for bash scripts and cron jobs with no code changes needed.

Full SDK docs at notilens.com/doc/sdk

The honest truth

You can't watch everything. Nobody on your team can.

But you can instrument the things that matter — revenue, user activity, scheduled jobs, agents — and let a system watch them for you.

The goal isn't a dashboard someone checks every morning. The goal is confidence that if something goes quiet or breaks, the right person finds out before your users do.

That's the only monitoring that matters at this stage.

NotiLens covers everything in this stack — silence detection, webhook monitoring, cron heartbeats, AI agent oversight, and automation monitoring. 7-day free trial, no credit card required.

We're giving eligible founders, small teams, and startups 3 months free in exchange for honest feedback — reach out directly if that's interesting.

Our SaaS stopped getting signups at 2am. No alerts fired. Here's why.

Stephen Souza — Tue, 05 May 2026 12:02:37 +0000

It was a Tuesday morning. I opened my laptop, checked the dashboard, and noticed something off.

No signups since 2:17am.

Not one. For almost six hours.

My first thought: slow day. Maybe people just weren't signing up. It happens.

My second thought, ten minutes later after staring at the chart: this has never happened before.

Everything was "up"

Here's the part that still bothers me.

My uptime monitor? Green.
Server health? Normal.
Error logs? Clean.
SSL cert? Valid.
API response time? Fine.

By every traditional measure, my product was working perfectly. Which meant none of my alerts fired. Not a single one.

But my signup flow was completely broken.

What actually happened

Somewhere around 2am, something in our signup flow started returning responses that looked like success — but weren't. Not a crash. Not an error. Just silent wrong behaviour. The kind that look like success to a health check but actually fail when a real user tries to sign up.

The signup form submitted. The spinner spun. Then nothing. No account created. No error shown to the user. Just a quiet dead end.

Users didn't get an error. They got silence. And they left.

For six hours.

I found out when a user emailed me saying they'd tried to sign up three times the night before and gave up.

That email hurt more than the outage itself.

The monitoring gap nobody talks about

Most monitoring tools are built around one question: "Is it up?"

Is the server responding? Is the endpoint returning 200? Is the cert valid?

These are good questions. But they're the wrong question for this failure.

The right question was: "Is anything actually happening?"

Specifically — are signups happening at the rate they normally do? Because on a normal Tuesday at 2am, even with low traffic, I get some signups. When that number drops to zero for six hours, something is wrong. Always.

But no tool I was using asked that question. They were all watching the pipes. Nobody was watching the water.

The difference between "up" and "working"

Your server can be up. Your API can respond. Your database can be connected.

And your business can still be silently bleeding.

This happens more than people admit:

Payment webhooks stop arriving — Stripe has a hiccup, webhooks queue and don't retry correctly. Your server never goes down. Your revenue processing stops.
Emails stop sending — your SMTP provider throttles you, but returns 200s. Onboarding emails never arrive. Users churn thinking you don't care.
Cron jobs silently skip — the job "runs" but processes zero records due to a config drift. No error. No alert. Your data pipeline is stale for days.
Signup flow breaks — exactly what happened to us. The form works. The backend "responds." Zero accounts created.
AI agents go quiet — agent is "running" but stops producing outputs. No exception, no crash. Task queue drains, nothing gets done.
AI agents loop infinitely — agent keeps retrying the same step, burning tokens and API credits silently. No alert. You see the bill at end of month.
AI agents get stuck — tool call hangs waiting for a response that never comes. The agent neither fails nor succeeds. Just... waits. Forever.

In every one of these cases, traditional monitoring sees nothing wrong. Because technically, nothing is wrong with the infrastructure. The failure is at the business event level — the things that actually matter.

What I wished I had

I didn't want another dashboard to stare at.

I wanted something to notice that signups had gone quiet — and tell me.

Not because I configured a threshold. Not because I manually set up an alert for "signups < 1 per hour." But because the system knew what normal looked like for my app at 2am on a Tuesday, and knew that zero signups for six hours was abnormal.

Basically: I wanted the monitoring equivalent of someone who's worked at my company long enough to say "hey, something feels off today."

What We built

After this incident — and two others like it in the same week (a CPU spike that correlated with a memory leak, and an anomalous jump in signups that turned out to be a bot wave) — We built NotiLens.

The core idea is what We call Smart Silence Detection.

Instead of asking "is the server up?", it asks "is your business behaving normally?"

It learns your baseline — what normal event volume looks like at each hour of the day, each day of the week. Then it alerts you when things go abnormally quiet. No manual threshold configuration needed. No spreadsheet of "expected events per hour." It just learns, and it watches.

When signups stop. When webhooks dry up. When your cron job runs but processes nothing. When orders drop to zero on a Saturday afternoon. When your AI agent stops producing outputs. When it starts looping through the same step burning your API credits. When it hangs waiting for a tool response that never comes.

That's the alert you need. Not "server down." But "your business just went silent."

There's a second pattern it catches that We didn't even plan for initially: broken flows.

Silence detection watches for events that stop happening altogether. Broken flow detection watches for events that start but never finish.

payment.initiated fired. But payment.completed never followed.

user.registered fired. But user.activated never followed.

order.placed fired. But order.fulfilled never followed.

Each individual event looks fine. No errors. No timeouts. The payment was "initiated" — technically true. But the money never moved.

This is where most revenue leaks actually happen. Not in crashes. In the gap between two events that should always travel together — but sometimes don't.

The practical setup (for the technically curious)

The way it works under the hood:

You send business events to NotiLens via a simple SDK call or webhook — signup.completed, payment.received, order.placed, whatever matters to your app.
NotiLens builds a rolling baseline of expected event frequency using ML — per event type, per hour of day, per day of week.
When observed frequency drops significantly below baseline for a sustained period, it fires an alert — push notification to your phone, Slack, email, whatever you have set up.
You also get anomaly detection in the other direction: sudden spikes (bot attacks, viral traffic, billing loops) are caught too.
For broken flows, you define the relationship between two events — payment.initiated should always be followed by payment.completed within X minutes. If it isn't, you get alerted immediately. No polling. No manual checks.

What I learned

A few things I'd tell myself before that Tuesday morning:

1. "No alerts" is not the same as "no problems."
Silence from your monitoring tools means your infrastructure is up. It says nothing about whether your business is working.

2. The failures that hurt most are the ones users experience silently.
A full server outage is obvious. Users tweet about it, you get flooded with emails, you know within minutes. A broken signup flow at 2am? You find out when a user emails you three days later — if they email at all. Most just leave.

3. Business events are first-class monitoring targets.
signup.completed, payment.received, user.activated — these deserve the same monitoring attention as CPU and memory. Maybe more.

4. Absence of data is data.
Zero signups for six hours is a signal. Treat it like one.

If any of this sounds familiar — if you've had that moment of "wait, when did this stop working?" — I'd love to hear your story in the comments.

And if you want to try NotiLens — if you're a solo founder, running a small team, building with AI agents, or just tired of juggling multiple systems with no single place watching whether your business is actually working — we're giving early users 3 months free in exchange for honest feedback. Just reach out directly or drop a comment below.