How to Set Up On-Call Monitoring and Alerts with Vigilmon

Getting notified when your site goes down is only half the picture. The other half is making sure the right person gets notified, through the right channel, at the right time - without alert fatigue. This guide covers how to configure Vigilmon's alerting system for effective on-call monitoring.

The Core Problem: Alert Routing

Many teams have monitoring but poor alerting. Their monitors fire, but:

• Alerts go to a shared inbox that nobody watches at 3 AM
• Alerts go to everyone, creating noise that gets ignored
• There's no escalation - if the first person misses it, nothing happens
• Alerts are too frequent or too sensitive, causing alert fatigue

Good monitoring without good alerting is useless. Here's how to set it up right.

Vigilmon's Alerting Options

Vigilmon supports multiple alert channels that you can combine:

1. Email alerts - immediate notification to one or more addresses
2. Webhook alerts - custom HTTP POST to any URL (Slack, Discord, PagerDuty, custom endpoints)
3. Alert delays - configure how long a monitor must be down before alerting (reduces false positives)
4. Recovery alerts - get notified when a monitor comes back up

Setting Up Email Alerts

The simplest starting point:

1. In Vigilmon, go to Notifications ? Email
2. Add the email address(es) to notify
3. Choose whether to alert on down, recovery, or both
4. Set a confirmation interval (e.g., re-alert if still down after 15 minutes)

For team setups: Create a team email alias (e.g., [e-mail]) that forwards to multiple engineers. This way you don't need to update Vigilmon every time someone joins or leaves the team.

Setting Up Slack Alerts

Slack is the most common destination for monitoring alerts in engineering teams.

Create a Slack Incoming Webhook

1. Go to your Slack workspace ? Apps ? Search "Incoming Webhooks"
2. Install the app and create a webhook for your #incidents or #alerts channel
3. Copy the webhook URL

Configure in Vigilmon

1. In Vigilmon, go to Notifications ? Webhook
2. Add your Slack webhook URL
3. Use this payload template:

{
  "text": "?? *{{monitor_name}}* is DOWN\n*Status:* {{status_code}}\n*URL:* {{monitor_url}}\n*Time:* {{timestamp}}\n*Location:* {{location}}",
  "username": "Vigilmon",
  "icon_emoji": ":rotating_light:"
}

For recovery alerts:

{
  "text": "? *{{monitor_name}}* is back UP\n*Downtime:* {{duration}}\n*Time:* {{timestamp}}",
  "username": "Vigilmon",
  "icon_emoji": ":white_check_mark:"
}

Setting Up Discord Alerts

For teams using Discord:

1. In your Discord server, go to channel settings ? Integrations ? Webhooks ? New Webhook
2. Copy the webhook URL
3. Append /slack to the Discord webhook URL - Discord accepts Slack-compatible payloads
4. Add the modified URL as a Vigilmon webhook

{
  "text": "?? {{monitor_name}} is DOWN - {{monitor_url}} returned {{status_code}}"
}

Integrating with PagerDuty

For teams with formal on-call rotations:

1. In PagerDuty, create a new service integration using the "Events API v2" integration
2. Copy your integration key
3. In Vigilmon, add a webhook pointing to https://events.pagerduty.com/v2/enqueue
4. Send the PagerDuty-formatted payload:

{
  "routing_key": "your-pagerduty-integration-key",
  "event_action": "trigger",
  "payload": {
    "summary": "{{monitor_name}} is DOWN - {{monitor_url}}",
    "severity": "critical",
    "source": "vigilmon",
    "timestamp": "{{timestamp}}"
  }
}

PagerDuty will then handle on-call scheduling, escalation, and phone alerts according to your configured policies.

Reducing False Positives with Alert Delays

Not every blip is an incident. A transient 500 error or a 10-second network hiccup doesn't warrant a 3 AM alert. Configure alert delays in Vigilmon:

• Set a confirmation period: Only alert if the monitor has been down for 2 consecutive checks (60 seconds with 30-second intervals)
• Use appropriate check intervals: For non-critical endpoints, 5-minute intervals reduce noise
• Set regional thresholds: Alert only if multiple regions confirm the outage (not just one)

Alert Severity Tiers

For production systems, separate your monitors by criticality:

Tier 1 - Immediate alert (30s interval, instant notification):

• Your homepage and primary signup/login flow
• Your payment processing endpoints
• Your API's main entry point

Tier 2 - Fast alert (1min interval, notify within 2 minutes):

• Secondary user-facing features
• Admin interfaces
• Key API endpoints

Tier 3 - Relaxed alert (5min interval, batch notification):

• Internal tools
• Staging environments
• Non-critical marketing pages

Route Tier 1 alerts to Slack #incidents + PagerDuty. Route Tier 2 to Slack #alerts. Route Tier 3 to email only.

Recovery Alerts

Always enable recovery alerts. When your site comes back up, you need to know:

1. The incident is over - stop investigating / wake up the on-call rotation
2. Total downtime duration - for post-mortems and SLA tracking
3. Recovery confirmation - to update your status page

Configure recovery alerts to go to the same channels as your down alerts.

Status Page: Alert Your Users Automatically

When an incident happens, your users need to know. Set up a Vigilmon status page:

1. Go to Status Pages ? New Status Page
2. Add your most critical monitors
3. Configure your domain: status.your-app.com
4. Share the URL in your app, docs, and email footer

When Vigilmon detects downtime, your status page reflects it automatically. Users can self-serve incident status instead of flooding your support inbox.

Sample Alert Flow for a Small SaaS Team

Monitor detects downtime (confirmed after 2 checks)
  ? Slack #incidents: immediate notification
  ? Email to on-call alias: immediate
  ? Status page: automatically shows incident

If still down after 10 minutes:
  ? PagerDuty page: escalate to on-call engineer

Monitor recovers:
  ? Slack #incidents: recovery notification with downtime duration
  ? Status page: automatically shows resolved
  ? Email: recovery confirmation

This flow ensures someone is always notified, escalation happens automatically if the first alert is missed, and users are informed without manual intervention.

Common Alerting Mistakes to Avoid

1. Alerting on every check failure - use confirmation periods to filter transient blips
2. Too many monitors at the same priority - not everything is critical; tier your alerts
3. No recovery alerts - you need to know when the incident is over
4. Alerts to a shared inbox - create a dedicated alias or use Slack channels
5. No status page - users will find out and email you anyway; give them a self-serve option

Summary

Effective on-call monitoring isn't just about detection - it's about the right person getting the right alert through the right channel with enough context to act immediately.

Recommended setup:

• Email: team alias for all alerts
• Slack #incidents: Tier 1 alerts (critical monitors)
• Slack #alerts: Tier 2 alerts
• PagerDuty/phone: If you have formal on-call rotations
• Status page: Always, for every user-facing product

?? Set up monitoring and alerting with Vigilmon - free tier available