How to Monitor Microservices: A Practical Guide for Distributed Systems
Monitoring a monolith is simple: one service, one health endpoint, one status page. Monitoring microservices is harder-you have dozens of services, each of which can independently fail, and failures can cascade in unexpected ways. This guide covers a practical approach to microservices monitoring with Vigilmon.
The Microservices Monitoring Challenge
In a microservices architecture, you face problems that don't exist in monoliths:
- Cascading failures: Service A fails because Service B is slow, because Service C is down
- Partial degradation: Some features work, others don't-users get inconsistent experiences
- External dependency failures: Third-party APIs your services depend on can fail
- Network failures: Services can be healthy but unable to reach each other
- Deployment drift: Service A was just deployed, Service B wasn't-compatibility issues
External uptime monitoring addresses the tip of the iceberg: you can tell which user-facing endpoints are degraded. Internal distributed tracing (Jaeger, Tempo, Zipkin) tells you why.
Layer 1: External Health Checks with Vigilmon
Each customer-facing service needs an external monitor. In Vigilmon, you monitor the entry points your users actually hit.
What to Monitor
For a typical e-commerce microservices app:
| Monitor | URL | What It Checks |
|---|---|---|
| Frontend | https://shop.example.com/health | Full UI stack |
| API Gateway | https://api.example.com/health | Routing layer |
| Auth service | https://api.example.com/auth/health | Login/signup |
| Product service | https://api.example.com/products/health | Catalog |
| Payment service | https://api.example.com/payments/health | Checkout |
| SSL cert | shop.example.com | Certificate expiry |
Aggregate Health Endpoint Pattern
The API Gateway should expose an aggregate health check that queries all downstream services:
` ypescript
// api-gateway/health.ts
import express from "express";
const app = express();
interface ServiceHealth {
name: string;
url: string;
status: "ok" | "error" | "timeout";
latencyMs: number;
}
async function checkService(name: string, url: string): Promise {
const start = Date.now();
try {
const response = await fetch(url, {
signal: AbortSignal.timeout(3000), // 3 second timeout
});
return {
name,
url,
status: response.ok ? "ok" : "error",
latencyMs: Date.now() - start,
};
} catch (error) {
return {
name,
url,
status: Date.now() - start >= 3000 ? "timeout" : "error",
latencyMs: Date.now() - start,
};
}
}
app.get("/health", async (req, res) => {
const services = [
{ name: "auth", url: "http://auth-service/health" },
{ name: "products", url: "http://product-service/health" },
{ name: "orders", url: "http://order-service/health" },
{ name: "payments", url: "http://payment-service/health" },
];
// Check all services in parallel
const results = await Promise.all(
services.map(({ name, url }) => checkService(name, url))
);
const allOk = results.every((r) => r.status === "ok");
const anyError = results.some((r) => r.status === "error");
res.status(allOk ? 200 : 503).json({
status: allOk ? "ok" : anyError ? "error" : "degraded",
timestamp: new Date().toISOString(),
services: Object.fromEntries(results.map((r) => [r.name, r])),
});
});
`
This gives Vigilmon a single endpoint to monitor that reflects the health of your entire stack.
Lean Health Endpoint Pattern
For individual services, keep health endpoints lightweight. Don't check downstream services-that creates circular dependency chains:
` ypescript
// auth-service/health.ts
// Only checks THIS service's dependencies (its own database, cache)
// Does NOT check other microservices
app.get("/health", async (req, res) => {
try {
await db.query("SELECT 1");
res.json({ status: "ok" });
} catch {
res.status(503).json({ status: "error" });
}
});
`
The aggregate health endpoint (at the gateway) checks downstream services. Individual services check only their own dependencies.
Layer 2: Internal Health for Service Discovery
In Kubernetes or Docker Swarm, internal health probes determine whether containers receive traffic:
Kubernetes Probes
`yaml
k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: product-service
livenessProbe:
httpGet:
path: /health/live
port: 3000
initialDelaySeconds: 10
periodSeconds: 15
failureThreshold: 3
readinessProbe:
httpGet:
path: /health/ready
port: 3000
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 2
`
` ypescript
// product-service/health.ts
// Liveness: Is the process running? (Simple-only fails for true crashes)
app.get("/health/live", (req, res) => {
res.json({ status: "ok" });
});
// Readiness: Is the service ready to receive traffic?
// (Checks DB, cache, etc.)
app.get("/health/ready", async (req, res) => {
try {
await db.query("SELECT 1");
await cache.ping();
res.json({ status: "ready" });
} catch (error) {
res.status(503).json({ status: "not_ready", error: String(error) });
}
});
`
Vigilmon monitors the external /health endpoint. Kubernetes uses /health/live and /health/ready for internal orchestration.
Layer 3: Cascading Failure Detection
Set up Vigilmon monitors with appropriate thresholds for cascading failure detection:
Fast detection for critical services (payments, auth):
- Interval: 1 minute
- Failure threshold: 1 (immediate alert)
- Regions: 2+ regions
Tolerant monitoring for non-critical services:
- Interval: 5 minutes
- Failure threshold: 2 (avoid false alarms during deploys)
- Regions: 1
Alert on Patterns, Not Just Failures
If 5 monitors fire simultaneously, it's likely a shared infrastructure issue (DNS, network, load balancer) rather than individual service failures. Configure your alert grouping accordingly:
In Vigilmon ? Alert settings:
- Group alerts by time window (alerts within 2 minutes = likely one incident)
- Route grouped alerts to PagerDuty with severity: critical
Layer 4: Dependency Health Checks
For third-party dependencies (Stripe, SendGrid, AWS S3), check their status APIs:
` ypescript
// external-health.ts
interface ExternalService {
name: string;
statusUrl: string;
}
const EXTERNAL_DEPS: ExternalService[] = [
{ name: "stripe", statusUrl: "https://status.stripe.com/api/v2/status.json" },
{ name: "sendgrid", statusUrl: "https://status.sendgrid.com/api/v2/status.json" },
];
app.get("/health/dependencies", async (req, res) => {
const checks = await Promise.allSettled(
EXTERNAL_DEPS.map(async (dep) => {
const response = await fetch(dep.statusUrl);
const data = await response.json();
return { name: dep.name, status: data.status?.indicator || "unknown" };
})
);
const results = checks.map((c, i) =>
c.status === "fulfilled"
? c.value
: { name: EXTERNAL_DEPS[i].name, status: "error" }
);
res.json({
timestamp: new Date().toISOString(),
dependencies: results,
});
});
`
Setting Up Vigilmon for Microservices
- Go to vigilmon.online
- Add monitors for each user-facing service endpoint
- Recommended monitor set:
- API Gateway aggregate health - 1-minute interval, 1 failure threshold
- Individual service health - 5-minute interval, 2 failure threshold
- SSL certificates - daily check, 30-day expiry alert
- Create a status page that groups services by category (Core, Payments, Notifications)
- Connect PagerDuty for critical service failures
Monitoring Architecture Summary
External users
?
Vigilmon (external uptime monitor)
? monitors
API Gateway /health (aggregate)
? checks
Service A /health Service B /health Service C /health
(own DB/cache) (own DB/cache) (own DB/cache)
?
Kubernetes probes (/health/live + /health/ready)
Summary
- External: Vigilmon monitors customer-facing endpoints
- Gateway: Aggregate health check queries all downstream services
- Individual: Each service checks only its own dependencies
- Kubernetes: Separate liveness/readiness probes for orchestration
- Use failure thresholds appropriately for critical vs. non-critical services
Monitor your microservices at vigilmon.online - free for 3 monitors, no credit card required.
Top comments (0)