Prometheus is the open-source monitoring system. Its client libraries let you instrument any Node.js app with custom metrics.
prom-client: The Node.js SDK
import { Registry, Counter, Histogram, Gauge, Summary, collectDefaultMetrics } from "prom-client";
const register = new Registry();
collectDefaultMetrics({ register }); // CPU, memory, event loop, etc.
// Counter — monotonically increasing
const scrapeCounter = new Counter({
name: "scrapes_total",
help: "Total number of scrape operations",
labelNames: ["source", "status"],
registers: [register],
});
scrapeCounter.inc({ source: "amazon", status: "success" });
scrapeCounter.inc({ source: "amazon", status: "error" });
// Histogram — request durations, response sizes
const scrapeDuration = new Histogram({
name: "scrape_duration_seconds",
help: "Scrape operation duration in seconds",
labelNames: ["source"],
buckets: [0.1, 0.5, 1, 2, 5, 10, 30],
registers: [register],
});
const end = scrapeDuration.startTimer({ source: "amazon" });
await scrapeProduct(url);
end(); // Records duration automatically
// Gauge — current value (up/down)
const activeScrapers = new Gauge({
name: "active_scrapers",
help: "Currently running scraper instances",
registers: [register],
});
activeScrapers.inc();
await scrape();
activeScrapers.dec();
// Summary — percentiles
const responseSize = new Summary({
name: "scrape_response_bytes",
help: "Scraped page size in bytes",
percentiles: [0.5, 0.9, 0.99],
registers: [register],
});
responseSize.observe(html.length);
Expose Metrics Endpoint
import express from "express";
const app = express();
app.get("/metrics", async (req, res) => {
res.set("Content-Type", register.contentType);
res.end(await register.metrics());
});
app.listen(3000);
Output:
# HELP scrapes_total Total number of scrape operations
# TYPE scrapes_total counter
scrapes_total{source="amazon",status="success"} 1523
scrapes_total{source="amazon",status="error"} 42
# HELP scrape_duration_seconds Scrape operation duration in seconds
# TYPE scrape_duration_seconds histogram
scrape_duration_seconds_bucket{source="amazon",le="0.1"} 100
scrape_duration_seconds_bucket{source="amazon",le="0.5"} 800
scrape_duration_seconds_bucket{source="amazon",le="1"} 1200
PromQL: Query Language
# Rate of scrapes per second
rate(scrapes_total[5m])
# Error rate
rate(scrapes_total{status="error"}[5m]) / rate(scrapes_total[5m])
# 99th percentile latency
histogram_quantile(0.99, rate(scrape_duration_seconds_bucket[5m]))
# Top 5 slowest sources
topk(5, avg by (source) (rate(scrape_duration_seconds_sum[5m]) / rate(scrape_duration_seconds_count[5m])))
Prometheus API: Query Programmatically
const result = await fetch('http://prometheus:9090/api/v1/query?query=rate(scrapes_total[5m])').then(r => r.json());
console.log(result.data.result);
// Range query
const range = await fetch('http://prometheus:9090/api/v1/query_range?' + new URLSearchParams({
query: 'rate(scrapes_total[5m])',
start: '2026-03-29T00:00:00Z',
end: '2026-03-29T23:59:59Z',
step: '60',
})).then(r => r.json());
Monitor scraping performance? My Apify tools + Prometheus = complete metrics.
Custom monitoring? Email spinov001@gmail.com
Top comments (0)