DEV Community

Vigilmon
Vigilmon

Posted on

How to Monitor Microservices: A Practical Guide for Distributed Systems

How to Monitor Microservices: A Practical Guide for Distributed Systems

Monitoring a monolith is simple: one service, one health endpoint, one status page. Monitoring microservices is harder-you have dozens of services, each of which can independently fail, and failures can cascade in unexpected ways. This guide covers a practical approach to microservices monitoring with Vigilmon.

The Microservices Monitoring Challenge

In a microservices architecture, you face problems that don't exist in monoliths:

  • Cascading failures: Service A fails because Service B is slow, because Service C is down
  • Partial degradation: Some features work, others don't-users get inconsistent experiences
  • External dependency failures: Third-party APIs your services depend on can fail
  • Network failures: Services can be healthy but unable to reach each other
  • Deployment drift: Service A was just deployed, Service B wasn't-compatibility issues

External uptime monitoring addresses the tip of the iceberg: you can tell which user-facing endpoints are degraded. Internal distributed tracing (Jaeger, Tempo, Zipkin) tells you why.

Layer 1: External Health Checks with Vigilmon

Each customer-facing service needs an external monitor. In Vigilmon, you monitor the entry points your users actually hit.

What to Monitor

For a typical e-commerce microservices app:

Monitor URL What It Checks
Frontend https://shop.example.com/health Full UI stack
API Gateway https://api.example.com/health Routing layer
Auth service https://api.example.com/auth/health Login/signup
Product service https://api.example.com/products/health Catalog
Payment service https://api.example.com/payments/health Checkout
SSL cert shop.example.com Certificate expiry

Aggregate Health Endpoint Pattern

The API Gateway should expose an aggregate health check that queries all downstream services:

` ypescript
// api-gateway/health.ts
import express from "express";

const app = express();

interface ServiceHealth {
name: string;
url: string;
status: "ok" | "error" | "timeout";
latencyMs: number;
}

async function checkService(name: string, url: string): Promise {
const start = Date.now();

try {
const response = await fetch(url, {
signal: AbortSignal.timeout(3000), // 3 second timeout
});

return {
  name,
  url,
  status: response.ok ? "ok" : "error",
  latencyMs: Date.now() - start,
};
Enter fullscreen mode Exit fullscreen mode

} catch (error) {
return {
name,
url,
status: Date.now() - start >= 3000 ? "timeout" : "error",
latencyMs: Date.now() - start,
};
}
}

app.get("/health", async (req, res) => {
const services = [
{ name: "auth", url: "http://auth-service/health" },
{ name: "products", url: "http://product-service/health" },
{ name: "orders", url: "http://order-service/health" },
{ name: "payments", url: "http://payment-service/health" },
];

// Check all services in parallel
const results = await Promise.all(
services.map(({ name, url }) => checkService(name, url))
);

const allOk = results.every((r) => r.status === "ok");
const anyError = results.some((r) => r.status === "error");

res.status(allOk ? 200 : 503).json({
status: allOk ? "ok" : anyError ? "error" : "degraded",
timestamp: new Date().toISOString(),
services: Object.fromEntries(results.map((r) => [r.name, r])),
});
});
`

This gives Vigilmon a single endpoint to monitor that reflects the health of your entire stack.

Lean Health Endpoint Pattern

For individual services, keep health endpoints lightweight. Don't check downstream services-that creates circular dependency chains:

` ypescript
// auth-service/health.ts
// Only checks THIS service's dependencies (its own database, cache)
// Does NOT check other microservices

app.get("/health", async (req, res) => {
try {
await db.query("SELECT 1");
res.json({ status: "ok" });
} catch {
res.status(503).json({ status: "error" });
}
});
`

The aggregate health endpoint (at the gateway) checks downstream services. Individual services check only their own dependencies.

Layer 2: Internal Health for Service Discovery

In Kubernetes or Docker Swarm, internal health probes determine whether containers receive traffic:

Kubernetes Probes

`yaml

k8s/deployment.yaml

apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: product-service
livenessProbe:
httpGet:
path: /health/live
port: 3000
initialDelaySeconds: 10
periodSeconds: 15
failureThreshold: 3
readinessProbe:
httpGet:
path: /health/ready
port: 3000
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 2
`

` ypescript
// product-service/health.ts

// Liveness: Is the process running? (Simple-only fails for true crashes)
app.get("/health/live", (req, res) => {
res.json({ status: "ok" });
});

// Readiness: Is the service ready to receive traffic?
// (Checks DB, cache, etc.)
app.get("/health/ready", async (req, res) => {
try {
await db.query("SELECT 1");
await cache.ping();
res.json({ status: "ready" });
} catch (error) {
res.status(503).json({ status: "not_ready", error: String(error) });
}
});
`

Vigilmon monitors the external /health endpoint. Kubernetes uses /health/live and /health/ready for internal orchestration.

Layer 3: Cascading Failure Detection

Set up Vigilmon monitors with appropriate thresholds for cascading failure detection:

Fast detection for critical services (payments, auth):

  • Interval: 1 minute
  • Failure threshold: 1 (immediate alert)
  • Regions: 2+ regions

Tolerant monitoring for non-critical services:

  • Interval: 5 minutes
  • Failure threshold: 2 (avoid false alarms during deploys)
  • Regions: 1

Alert on Patterns, Not Just Failures

If 5 monitors fire simultaneously, it's likely a shared infrastructure issue (DNS, network, load balancer) rather than individual service failures. Configure your alert grouping accordingly:

In Vigilmon ? Alert settings:

  • Group alerts by time window (alerts within 2 minutes = likely one incident)
  • Route grouped alerts to PagerDuty with severity: critical

Layer 4: Dependency Health Checks

For third-party dependencies (Stripe, SendGrid, AWS S3), check their status APIs:

` ypescript
// external-health.ts
interface ExternalService {
name: string;
statusUrl: string;
}

const EXTERNAL_DEPS: ExternalService[] = [
{ name: "stripe", statusUrl: "https://status.stripe.com/api/v2/status.json" },
{ name: "sendgrid", statusUrl: "https://status.sendgrid.com/api/v2/status.json" },
];

app.get("/health/dependencies", async (req, res) => {
const checks = await Promise.allSettled(
EXTERNAL_DEPS.map(async (dep) => {
const response = await fetch(dep.statusUrl);
const data = await response.json();
return { name: dep.name, status: data.status?.indicator || "unknown" };
})
);

const results = checks.map((c, i) =>
c.status === "fulfilled"
? c.value
: { name: EXTERNAL_DEPS[i].name, status: "error" }
);

res.json({
timestamp: new Date().toISOString(),
dependencies: results,
});
});
`

Setting Up Vigilmon for Microservices

  1. Go to vigilmon.online
  2. Add monitors for each user-facing service endpoint
  3. Recommended monitor set:
    • API Gateway aggregate health - 1-minute interval, 1 failure threshold
    • Individual service health - 5-minute interval, 2 failure threshold
    • SSL certificates - daily check, 30-day expiry alert
  4. Create a status page that groups services by category (Core, Payments, Notifications)
  5. Connect PagerDuty for critical service failures

Monitoring Architecture Summary


External users
?
Vigilmon (external uptime monitor)
? monitors
API Gateway /health (aggregate)
? checks
Service A /health Service B /health Service C /health
(own DB/cache) (own DB/cache) (own DB/cache)
?
Kubernetes probes (/health/live + /health/ready)

Summary

  • External: Vigilmon monitors customer-facing endpoints
  • Gateway: Aggregate health check queries all downstream services
  • Individual: Each service checks only its own dependencies
  • Kubernetes: Separate liveness/readiness probes for orchestration
  • Use failure thresholds appropriately for critical vs. non-critical services

Monitor your microservices at vigilmon.online - free for 3 monitors, no credit card required.

Top comments (0)