DEV Community: 우병수

AWS vs. Home Lab for a DevOps Startup: When Cloud Costs More Than the Hardware

우병수 — Mon, 10 Aug 2026 08:11:04 +0000

TL;DR: The moment that forces the AWS-vs-homelab decision usually isn't a planning meeting — it's a billing alert at the end of month three. A single `t3.

📖 Reading time: ~22 min

What's in this article

The Bill That Forces the Decision
What You're Actually Comparing (Constraints First)
Running the DevOps Stack on Home Lab Hardware
Running the Same Stack on AWS
Side-by-Side: Where Each Option Wins and Loses
Hybrid Architecture: The Practical Middle Ground
When to Pick What: Decision Rules Per Situation

The Bill That Forces the Decision

The moment that forces the AWS-vs-homelab decision usually isn't a planning meeting — it's a billing alert at the end of month three. A single t3.xlarge running continuously costs roughly $120/month in compute alone, but that's the floor, not the ceiling. Stack a 100GB gp3 EBS volume (~$8), a NAT gateway sitting idle most of the day (~$32 base + $0.045/GB processed), and even modest data transfer out, and you're looking at $200–250/month for one instance doing staging work. Now add a second runner for CI, a small RDS instance for your staging database, and a container registry with frequent pulls, and the number doubles without a single production user touching the system.

A real DevOps lab has a predictable surface area: GitLab or Gitea runners pulling jobs, a private container registry (Harbor or a plain registry:2 container), at least one staging environment that mirrors prod, a secrets backend (Vault or something lighter), and a monitoring stack — Prometheus, Grafana, Loki, or whatever subset you trust. Every single one of those has a corresponding AWS line item. ECR charges per GB stored and per GB transferred. Secrets Manager charges per secret per month plus per 10,000 API calls. CloudWatch log ingestion is $0.50/GB and retrieval is priced separately. None of these amounts individually sounds alarming, which is exactly why the composite bill surprises people.

Egress is the cost category that almost no one budgets correctly at the start. A staging environment that pulls a 2GB Docker image on every runner warmup, ships structured logs to an external sink, and calls a third-party API through a NAT gateway is generating egress charges on multiple vectors simultaneously. AWS charges $0.09/GB for data transferred out to the internet (in us-east-1 as of current pricing). A CI pipeline that builds and pushes a 1.5GB image, then pulls it into a staging cluster, then ships 500MB of logs — run that 20 times a day across a small team and the egress line alone clears $80–100/month. That number isn't on any pricing calculator because nobody types "20 CI runs × 2GB" into the estimator.

The amortization math on owned hardware is simpler than people expect once you force yourself to do it. A used Dell PowerEdge R730 with 64GB RAM and a pair of decent SSDs runs $400–700 on eBay. A new mini-PC like a Beelink SER7 (Ryzen 9 7940HS, 32GB RAM, 1TB NVMe) is about $350 retail. Either machine runs your entire DevOps lab — runners, registry, staging containers, Vault, Prometheus — at a fixed electricity cost of roughly $15–30/month depending on load and local rates. The cloud equivalent of that lab costs $400–600/month on AWS with no ceiling. The hardware pays for itself in under two months. The honest counter-argument is ops burden and the time cost of maintaining the box, which is real — but for teams already running self-hosted automation stacks, that overhead is already priced in. The broader tooling picture for that kind of setup is covered in our Workflow Automation in 2026: n8n, Zapier, and Self-Hosted Pipelines guide.

What You're Actually Comparing (Constraints First)

The framing of "cloud vs. home lab" obscures the real trade-off, which is this: do you want your budget risk to show up as a one-time capital hit you can plan for, or as a variable invoice that spikes whenever something goes wrong or traffic does something unexpected? AWS doesn't punish you for building — it punishes you for forgetting. A runaway Lambda, a CloudWatch log group with no retention policy, an NAT Gateway sitting in front of three t3.micro instances — these are the actual cost vectors, and none of them are obvious until you've already paid for them.

The home lab baseline is more concrete than people expect. A used enterprise tower — something like a Dell PowerEdge R740 or a Supermicro workstation with a Ryzen 9 5950X or Xeon Silver, 64–128 GB ECC RAM, and a pair of NVMe drives in a software RAID — runs between $800 and $2,000 depending on what you're willing to spec. That machine runs Proxmox with no licensing fee, hosts a full GitLab instance (CI runners included), a Docker Swarm or single-node Kubernetes cluster, and a local container registry — all simultaneously, all without per-minute billing. The disk I/O on NVMe is faster than most EBS gp3 volumes at equivalent workloads. Power draw is real (roughly 80–150W under moderate load), but at residential electricity rates that's $7–15/month, not a line item that changes your runway.

The AWS baseline for a comparable DevOps lab isn't one service — it's a composition. You're looking at EC2 (t3.medium to m6i.large depending on what you're running), ECR for storing container images (cheap per GB, but egress adds up), S3 for build artifacts and Terraform state, CloudWatch for logs with retention you have to configure manually or pay for indefinitely, and IAM roles you'll spend a non-trivial afternoon debugging. Each of those is priced reasonably in isolation. Together, without deliberate cost management, a three-person startup running daily CI builds, staging environments, and some observability tooling can hit $400–800/month before they've shipped anything to production.

The constraint that actually forces the decision isn't technical — it's risk tolerance. A home lab failure (dead drive, power event, network hiccup) means downtime you have to fix yourself, on your schedule, with your hands. An AWS failure (or more commonly, an AWS misconfiguration) means an invoice you can't reverse, or an IAM policy that silently breaks your pipeline at 2am. Startups that are pre-revenue and have someone technical on-call 24/7 can absorb the former. Startups with investors, SLAs, or a team that needs to ship and not babysit infrastructure lean toward the latter — not because AWS is better engineering, but because managed failure recovery has a dollar cost that's easier to budget than a time cost that lands on whoever is available. Neither answer is wrong; they're just answers to different questions about where you want your operational pain to live.

Running the DevOps Stack on Home Lab Hardware

The part most cloud-first engineers underestimate is how complete a home lab stack can actually be. Gitea is about 80MB of RAM at idle and gives you webhooks, pull requests, and fine-grained access tokens — everything a team of one to five needs from a Git host. Self-hosted GitLab CE is the heavier option: expect 3-4GB RAM minimum before you add runners, but you get built-in container registry, CI, and merge request pipelines in a single deployment. For most home lab workloads, Gitea paired with a separate CI tool is the better call on constrained hardware.

For pipelines, Woodpecker CI connects directly to Gitea or GitHub via OAuth and runs jobs in Docker containers — the config is a .woodpecker.yml in the repo root, and the agent is stateless, so you can run multiple agents on different machines without coordination. Forgejo Actions is worth knowing if you're already running Forgejo (the Gitea fork): it accepts GitHub Actions YAML syntax, which means migrating existing workflows costs almost nothing. Both tools use Docker socket or a Docker daemon for job isolation, which does mean a compromised pipeline step can touch the host — plan your network segmentation accordingly.

`yaml

Woodpecker agent — minimal compose fragment

woodpecker-agent:
image: woodpeckerci/woodpecker-agent:v2.7.0
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WOODPECKER_SERVER=grpc://woodpecker-server:9000
- WOODPECKER_AGENT_SECRET=${AGENT_SECRET}
- WOODPECKER_MAX_WORKFLOWS=4 # tune to CPU core count
`

Harbor gives you a private OCI registry with vulnerability scanning (Trivy backend), image replication, and RBAC — deployed via Docker Compose, it lands around 600-800MB RAM across its services. The alternative is running a plain registry:2 container, which is leaner but gives you nothing beyond push/pull. Harbor is worth the overhead if you're doing multi-stage builds and want to catch CVEs before images reach your staging environment. Pair it with Grafana + Prometheus + Loki for observability: Prometheus scrapes your services, Loki ingests logs shipped via Promtail or the Docker logging driver, and Grafana ties it together. On modest hardware, keep Loki's retention window short and configure chunk caching or you'll watch it eat disk faster than expected.

Proxmox as the hypervisor layer is where home lab genuinely beats a comparable cloud budget. Snapshot-based VM cloning means your staging environment is a full copy of production state — no AMI bake time, no S3 transfer costs. The command to clone VM 100 into VM 200 with a full independent disk copy is:

`shell

Full clone — produces a completely independent VM from template ID 100

qm clone 100 200 --full --name staging-env-01

Snapshot before destructive test

qm snapshot 200 pre-test --description "before chaos run"

Roll back if needed

qm rollback 200 pre-test
`

That snapshot workflow is the rough functional equivalent of launching from an EC2 AMI and terminating on failure — except the round trip is seconds, not minutes, and you're not paying per-API-call or per-GB of AMI storage. For reverse proxy and TLS, Caddy handles wildcard certs from Let's Encrypt with DNS challenge in about a dozen lines of config — no listener rules, no target group gymnastics:

`nginx

Caddyfile — wildcard TLS for all internal services on one IP

*.lab.yourdomain.com {
tls {
dns cloudflare {env.CF_API_TOKEN}
}
@gitea host gitea.lab.yourdomain.com
handle @gitea {
reverse_proxy gitea:3000
}
@harbor host harbor.lab.yourdomain.com
handle @harbor {
reverse_proxy harbor-nginx:8080
}
}
`

The honest failure point is Postgres. Every managed RDS feature you take for granted — point-in-time recovery, multi-AZ failover, automated minor version patching — you now own entirely. A minimal trustworthy setup requires at minimum: WAL archiving to a separate disk or NAS (configure archive_command in postgresql.conf), a tested restore procedure you've actually run end-to-end, and a monitoring alert on replication lag if you add a replica. Patroni or repmgr can handle automated failover, but both have real operational complexity. Budget several hours to get this right before you route anything important through it — the "just run Postgres in Docker" path works until your drive fails at 2am and you realize your last verified backup is three weeks old.

Running the Same Stack on AWS

The honest case for AWS isn't that it's cheaper — it's that the first week goes smoother. RDS Multi-AZ handles failover without you writing a single line of HA logic. ECS Fargate means you describe a container and it runs, no node pool to patch, no kubelet to debug at 11pm. ECR lifecycle policies let you write one JSON block and forget about disk pressure from accumulated image layers. CloudWatch alarms are wired to IAM and SNS out of the box. None of that is magic, but it does represent real operational complexity that you're offloading. The trap is mistaking "easier at the start" for "cheaper to run long-term" — and that mistake usually surfaces around month three, when the bill arrives with line items you didn't anticipate.

Terraform reproducibility is the strongest real argument for AWS, and it's worth being specific about why. On AWS, terraform destroy && terraform apply actually works as a staging reset workflow. You can blow away an entire environment — VPC, subnets, ECS services, RDS instance, load balancer — and rebuild it in 12-15 minutes. On Proxmox or bare metal, doing the same thing requires you to have invested upfront in templating discipline: cloud-init images, Ansible playbooks, and a willingness to actually run them instead of SSHing in and fixing things by hand. Most home lab operators have a Terraform state file that describes what they intended to build, not what's actually running. AWS keeps that gap smaller because the API is the only way to make changes.

`json

Real lifecycle policy that prevents ECR from eating your storage budget

Without this, a busy CI pipeline can accumulate hundreds of GB in weeks

aws ecr put-lifecycle-policy \
--repository-name my-app \
--lifecycle-policy '{
"rules": [{
"rulePriority": 1,
"description": "Expire untagged images after 7 days",
"selection": {
"tagStatus": "untagged",
"countType": "sinceImagePushed",
"countUnit": "days",
"countNumber": 7
},
"action": { "type": "expire" }
},{
"rulePriority": 2,
"description": "Keep only last 20 tagged images",
"selection": {
"tagStatus": "tagged",
"tagPrefixList": ["v"],
"countType": "imageCountMoreThan",
"countNumber": 20
},
"action": { "type": "expire" }
}]
}'
`

The surprise line items follow a pattern. NAT Gateway is the most common one — at $0.045 per GB processed (verify against current AWS pricing; this changes), a staging environment where your containers pull Docker images through a NAT Gateway or ship verbose logs to CloudWatch can generate meaningful data-transfer charges without anyone noticing. CloudWatch Logs ingestion runs $0.50 per GB ingested, and if your Node.js app is logging every HTTP request at DEBUG level, a moderately trafficked service generates more log volume than most people expect. ECR storage is $0.10/GB/month — cheap until your CI runs 50 builds a day and you didn't set the lifecycle policy above. None of these are architectural problems; they're configuration problems that cost money while you figure them out.

Spot and Graviton instances are where AWS actually competes on price for CI workloads specifically. A c7g.2xlarge (Graviton3, 8 vCPU, 16GB RAM) on Spot can run well under half the On-Demand price during off-peak hours in most regions — useful if your build jobs run at predictable times. The catch is that Spot interruptions are not handled automatically by GitHub Actions, GitLab CI, or most other CI tools. You need either an interruption-aware runner wrapper (AWS provides one for CodeBuild; for GitHub Actions you're looking at terraform-aws-github-runner or similar) or you accept that a 2-minute warning and a failed build is acceptable. For stateless, idempotent build jobs it usually is. For anything with shared state — a Selenium grid, a database integration test — Spot interruption mid-run is a real failure mode, not a theoretical one.

Side-by-Side: Where Each Option Wins and Loses

The decision almost always gets made on the wrong axis. Teams compare sticker prices — a t3.medium vs. what electricity costs — and miss the dimensions that actually blow up months later: how long it takes a new engineer to get their first pipeline green, or what happens at 2am when your single physical host drops off the network. Here's the honest breakdown across the dimensions that matter.

Dimension

Home Lab

AWS

Monthly cost floor

Hardware already amortized → electricity + ISP only. Predictable.

~$50–200/mo minimum for anything resembling a real CI stack. Scales with mistakes.

Disaster recovery complexity

Single site. No automatic failover. Manual recovery unless you've built a second node yourself.

Multi-AZ is a config option. RDS automated backups, EBS snapshots — all managed.

Environment reproducibility

Identical if you're disciplined with NixOS or Ansible. Drifts badly if you're not.

AMI + Terraform = repeatable by default. New region spins up identically.

Compliance audit readiness

You write every control doc yourself. Auditors want evidence trails you have to build.

CloudTrail, IAM, GuardDuty, and existing SOC 2 reports from AWS cover most control gaps.

Internet dependency

Outbound only for external pulls. Internal pipelines survive ISP outages.

Everything routes through AWS endpoints. ISP down = engineers blocked.

Time-to-first-pipeline (new engineer)

VPN config + SSH key + runner registration. 30–90 minutes if docs exist.

IAM console + CodeBuild or GitHub Actions OIDC. 15–30 minutes with decent IaC.

Home lab wins in two specific scenarios and loses everywhere else. The first is sustained compute-heavy workloads where the cost-per-hour gap becomes obscene at scale — if you're running embedding pipelines against a corpus all day, every day, a local bge-m3 instance on a 32GB VRAM card costs you electricity while the equivalent p3 instance on AWS bills by the second. The second is anything requiring full network layer control: custom routing, promiscuous mode for packet capture, GPU passthrough configs that EC2 won't expose. You own the hardware, you own the kernel, you own the NIC — and that matters when your internal tooling does LLM inference on sensitive data that cannot leave the building.

AWS wins cleanly on burst capacity and compliance story. A startup with a monorepo that occasionally triggers 40 parallel test runners needs elastic capacity it genuinely cannot predict. Spot instances absorb that burst without you buying hardware that sits idle 90% of the time. On the compliance side: if a SOC 2 Type II auditor is asking for evidence of least-privilege access controls and audit logging, you can point at CloudTrail and IAM policy attachments. Reproducing that paper trail from a self-hosted setup requires building it yourself — a Gitea instance plus Vault plus structured syslog plus a policy-as-code layer — which is doable but is a significant chunk of engineering time that most early-stage teams don't have.

The dealbreaker for each side is predictable once you've run both long enough. Home lab's dealbreaker is physical single-site failure. A PSU dies, a switch port fails, ISP goes dark for six hours — there's no automatic failover. You can mitigate with a Tailscale-connected second node at a co-lo or a friend's rack, but that's still manual topology work. AWS's dealbreaker is cost unpredictability from human error: a junior engineer ships verbose CloudWatch logging on a high-throughput service, forgets to scope the log retention, and the bill for that month arrives with a line item that looks like a typo. Or a p3.2xlarge gets left running over a holiday weekend because the auto-shutdown Lambda wasn't wired up. Neither scenario is AWS's fault, but the blast radius is a bill you can't negotiate down. On a home lab, a runaway process costs you CPU cycles and maybe a thermal event — not a four-figure invoice.

Choose home lab when: workloads are sustained and predictable, data sensitivity demands air-gap-adjacent control, or VRAM-heavy inference is a daily operational need — not a burst use case.
Choose AWS when: CI load is spiky and unpredictable, a compliance framework is an active customer requirement, or onboarding speed matters more than cost optimization right now.
Hybrid is often the honest answer: home lab hosts the always-on services (Gitea, registry, monitoring, local LLM), AWS absorbs burst CI runners via OIDC-connected ephemeral instances that terminate on job completion.

Hybrid Architecture: The Practical Middle Ground

The false binary — cloud vs. home lab — dissolves the moment you stop treating each as a complete answer and start treating them as layers. The actual constraint is type of workload: stateful services, GPU compute, and internal tooling have a fundamentally different cost profile than public-facing endpoints or short-lived burst jobs. Once you wire them together properly, you stop making emotional infrastructure decisions and start making economic ones.

WireGuard as the Backbone

A site-to-site WireGuard tunnel between your home lab and a small AWS VPC (a t3.micro NAT instance works fine as the peer) is what makes the hybrid model operational rather than theoretical. You bring up the tunnel with wg-quick up wg0 on both sides, add a static route in the VPC route table pointing your home lab's subnet (say 10.10.0.0/24) at the WireGuard peer's private IP, and from that point on your Terraform-managed cloud resources can reach internal services by private IP — no public exposure, no bastion theater. A minimal home lab wg0.conf looks like this:

`ini
[Interface]
Address = 10.10.0.1/24
PrivateKey =
ListenPort = 51820

[Peer]

AWS WireGuard peer (t3.micro in the VPC)

PublicKey =
Endpoint = :51820
AllowedIPs = 172.16.0.0/24 # VPC private subnet
PersistentKeepalive = 25
`

On the AWS side, add 172.16.0.0/24 → eni-xxxx in your VPC route table and make sure the WireGuard instance's security group allows UDP 51820 inbound from your home IP. The latency over this tunnel is real — expect 10–30ms depending on your ISP — but for internal service calls (hitting a local Postgres, a local model inference endpoint, a private registry) that's irrelevant. What matters is that the traffic never touches the public internet and your home lab doesn't need an open inbound port beyond the WireGuard UDP port.

Routing CI Jobs by Workload Type, Not by Convention

The most immediate cost win in a hybrid setup is splitting CI traffic by job character rather than running everything in one place. Commit-triggered unit tests belong on self-hosted Woodpecker runners: they hit the local Docker cache, finish in seconds, and cost nothing per run. Release builds and security scans are a different shape entirely — they need clean environments, they're infrequent enough that keeping a warm instance for them is waste, and security scan tools (Trivy, Semgrep, OWASP dependency-check) pull large DBs you don't want polluting your local cache. Route those to AWS Spot c6i instances via a cloud-hosted Woodpecker agent or GitHub Actions with a self-hosted runner registered against an EC2 Spot Fleet. The Woodpecker pipeline split looks like this:

`yaml
steps:
unit-tests:
image: node:20-alpine
# runs on local runner — fast cache, no AWS cost
when:
event: push

security-scan:
image: aquasec/trivy:latest
commands:
- trivy fs --exit-code 1 --severity HIGH,CRITICAL .
# runs on AWS Spot runner — clean env, burst capacity
when:
event: tag
environment:
RUNNER_LABELS: aws-spot
`

The practical gotcha: Spot interruptions during a release build will fail your pipeline at the worst moment. Use --interruption-behavior terminate with a retry policy, or use Spot with an On-Demand fallback via a mixed instances policy. For builds that run less than 15 minutes, interruption rates on c6i in us-east-1 are low enough that a single retry covers almost every case.

Cost Monitoring on Both Sides Without Exceptions

A hybrid setup doubles the surface area for cost surprises — AWS bills you for things you forgot to deprovision, and the home lab quietly runs workloads that should have been stopped months ago. Treat monitoring as mandatory infrastructure, not a nice-to-have. On the AWS side, set an SNS-backed Budget alert at 80% of your monthly threshold — this is three CLI commands:

`shell

Create the budget (replace ACCOUNT_ID and EMAIL)

aws budgets create-budget \
--account-id ACCOUNT_ID \
--budget '{"BudgetName":"monthly-cap","BudgetLimit":{"Amount":"150","Unit":"USD"},"TimeUnit":"MONTHLY","BudgetType":"COST"}' \
--notifications-with-subscribers '[{"Notification":{"NotificationType":"ACTUAL","ComparisonOperator":"GREATER_THAN","Threshold":80,"ThresholdType":"PERCENTAGE"},"Subscribers":[{"SubscriptionType":"EMAIL","Address":"you@example.com"}]}]'
`

On the home lab side, a cron job that runs every 15 minutes and dumps docker stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}" plus df -h / output to a local log file — and alerts you if any container is above a CPU threshold for more than three consecutive checks — catches the runaway container that's been eating 100% of a core since Tuesday. Keep both alert systems independent: if AWS goes over budget, you should know regardless of whether your home lab monitoring is healthy, and vice versa. The goal isn't dashboards — it's no surprises on either invoice.

When to Pick What: Decision Rules Per Situation

The mistake most small teams make is treating this as a philosophical question about cloud vs. on-prem when it's actually an operational capacity question. You don't pick AWS because it's "more professional" or home lab because you're "scrappy." You pick based on three things: your team's Linux tolerance, your workload's variance, and what your contracts or customers require of you. Everything else is noise.

Go home lab first if your team is one to three people and your workloads are predictable. Predictable means you can answer "how much compute do we need at 2pm on a Tuesday?" with reasonable confidence. If you're already running Docker Compose services locally — a Postgres container, an n8n instance, maybe a local registry — you already understand the operational surface. The jump to running those same containers on a bare-metal server under your desk or in a rack at a colocation facility is smaller than it looks. The overhead is real: disk failures happen, UPS batteries die, you will eventually lose a Saturday to a kernel panic. But it's bounded overhead. You know all the failure modes after a few months, and they stop surprising you. The one hard requirement: someone on the team needs to be genuinely comfortable with Linux — not "can Google a command" comfortable, but "can diagnose why a container won't start after a reboot without a Stack Overflow session" comfortable.

Go AWS first under three specific conditions. First, if your customers or investors require SOC 2 Type II or HIPAA controls. The audit trail, the managed IAM, the compliance documentation that AWS generates automatically — reproducing that on home lab hardware costs more in engineering time than just paying AWS. Second, if your CI workload is spiky and unpredictable. Batch processing that runs fine on eight cores most of the time but occasionally needs 64 cores for a model training run is exactly what AWS spot instances exist for. Home lab hardware is sized for your median load; AWS lets you pay for your peaks. Third, if your team cannot tolerate a failed build server during a demo or a customer call. The emotional cost of that moment is real, and if your organization doesn't have the culture to shrug it off, don't build the system that creates it.

Go hybrid immediately if you're running local LLM inference for internal developer tooling. This is a genuinely useful workload to keep on owned hardware: embedding-based semantic search over internal docs, code review assistance, changelog summarization — all of these run fine on a 32GB VRAM box with bge-m3 for embeddings and a mid-size Ollama model for generation. The inference cost on AWS would make these tools economically stupid to operate. But if those same developers need public-facing APIs, a CDN, or anything that requires a stable IP and uptime SLA for external users, that piece belongs on AWS or a VPS. Split the workload at the network boundary: internal tools stay local, external surface goes cloud. The routing overhead is minimal if you're already on Tailscale.

The clearest migration signal is financial, and it's not ambiguous: if your AWS bill exceeds the annualized cost of equivalent home lab hardware for three consecutive months, the arithmetic has already answered the question. A workstation with a mid-range GPU, 128GB of RAM, and 8TB of NVMe runs somewhere between $3,000 and $6,000 depending on specs. If you're paying more than $250–500/month on AWS for equivalent compute capacity — and staying there month after month — you're past the crossover point. The remaining variable isn't financial, it's operational: is your team willing to own the hardware? If yes, migrate. If no, you're paying a management tax and at least you're paying it consciously.

Disclaimer: This article is for informational purposes only. The views and opinions expressed are those of the author(s) and do not necessarily reflect the official policy or position of Sonic Rocket or its affiliates. Always consult with a certified professional before making any financial or technical decisions based on this content.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

5 Self-Hosted Analytics Tools You Can Actually Run on Your Own Server

우병수 — Fri, 07 Aug 2026 08:10:51 +0000

TL;DR: GA4's data sampling kicks in at surprisingly low traffic levels — and the threshold isn't based on what would be statistically useful for you, it's based on what's computationally cheap for Google. A small e-commerce site running a conversion funnel report over a 90-day window w

📖 Reading time: ~20 min

What's in this article

The Problem With Sending Your Traffic Data Somewhere Else
How to Read This Comparison
The Five Tools: Setup Snapshots and Real Trade-offs
Deployment Configs Worth Copying
Comparison Table: What Matters at the Operator Level
When to Pick Which Tool
Keeping These Running: Backup and Upgrade Notes

The Problem With Sending Your Traffic Data Somewhere Else

GA4's data sampling kicks in at surprisingly low traffic levels — and the threshold isn't based on what would be statistically useful for you, it's based on what's computationally cheap for Google. A small e-commerce site running a conversion funnel report over a 90-day window will hit sampled results long before a high-traffic publisher does, which means the sites with the fewest data points to begin with are the ones getting their data thinned out further. The 14-month retention limit compounds this: any cohort or funnel analysis that crosses that window is just gone, not archived somewhere you could pay to access — gone.

The tracker-blocking problem is worse than most analytics dashboards will tell you. uBlock Origin, Brave's built-in shields, Firefox's Enhanced Tracking Protection, and DNS-level blockers like Pi-hole all drop google-analytics.com and gtag calls by default. What you see in GA4 is an undercount, and the undercount is skewed: technical audiences, privacy-conscious users, and anyone on a managed corporate network are disproportionately invisible. Beyond the measurement gap, every pageview that does get through is a data point handed to an advertising platform with its own interests in how that data gets used and retained.

Running your own analytics instance changes the data ownership model entirely. Your events land in a Postgres or ClickHouse table that you control — no sampling algorithm between collection and query, no retention policy you didn't write, and in many jurisdictions no cookie consent banner required at all when visitor data never leaves your infrastructure. The EU's GDPR guidance, and similar frameworks, generally treat server-side collection that doesn't involve cross-site tracking or external processors differently from GA4, which explicitly routes data through Google's servers. That's not legal advice, but it's a real architectural difference worth understanding before you dismiss self-hosted analytics as overkill.

What follows covers five tools that actually run in Docker without heroic effort: Plausible, Umami, Matomo, PostHog, and Fathom Lite. For each one, you'll get an honest read on RAM and disk growth over time — because "lightweight" means different things at 10k monthly visits versus 500k — along with the specific use case it fits best and where it breaks down. Some of these are genuinely impressive. Some have sharp edges that only show up after a few weeks of real traffic. The goal is to give you enough to make the right call for your setup, not to sell you on self-hosting as a philosophy.

How to Read This Comparison

Every comparison like this has hidden assumptions baked in. Here they're explicit: all five tools were evaluated under operator constraints that match a typical solo or small-team deployment — a single VPS or home-lab host, Docker Compose for orchestration, and a reverse proxy handling TLS (either Caddy with automatic certs or Nginx with manual config). No managed RDS, no cloud-hosted Postgres, no autoscaling. If a tool's architecture assumes it can phone home to a managed service or spin up additional nodes, that counts against it here.

The resource numbers you'll see reflect steady-state behavior at moderate traffic — roughly 50k pageviews per month. That's not a high-traffic site, but it's enough to expose tools that bloat their write queues or hold too much in memory between flushes. Where cold-start behavior is meaningfully different (some tools take 30–90 seconds before they'll accept ingestion), or where a traffic spike causes visible degradation, those are called out explicitly rather than buried in a footnote.

The comparison table uses five columns that actually matter for operational decisions:

Minimum RAM — what the container actually consumes under the load profile above, not the optimistic number from the README
Database backend — because your backup strategy, query performance, and migration path all depend on this
Data export format — CSV, JSON, and Parquet are not equivalent; one of these tools gives you neither CSV nor JSON by default
Multi-site support — whether you can track multiple domains under one install, or need separate deployments
Single biggest operational gotcha — the thing that isn't in the README but will find you after two weeks of running it in production

One framing note: this comparison covers pure analytics tools — event tracking, pageviews, funnels, session data. It doesn't cover self-hosted experimentation platforms, full observability stacks, or AI-assisted content pipelines. If you're thinking about how local-model tooling fits into the broader self-hosted picture, the AI Coding Tools in 2026: Cloud Copilots vs Local Models guide covers how those pieces connect in a real stack.

The Five Tools: Setup Snapshots and Real Trade-offs

The most common mistake when evaluating these tools is treating them as interchangeable. They aren't. The gap between Ackee and PostHog isn't just features — it's three orders of magnitude difference in infrastructure complexity. Pick based on what you're actually tracking, not what sounds impressive.

Plausible Analytics (Community Edition)

The Docker Compose setup is genuinely one command, and the plausible/analytics image bundles everything including the ClickHouse dependency. The gotcha that isn't in the README: ClickHouse has no memory cap by default and will cheerfully consume available RAM on a shared VPS as your event volume grows. You need to drop a config file into the container before this bites you:

# clickhouse-config.xml — mount this at /etc/clickhouse-server/config.d/
<yandex>
  <max_memory_usage>512000000</max_memory_usage>          <!-- 512MB hard cap -->
  <max_memory_usage_for_all_queries>800000000</max_memory_usage_for_all_queries>
</yandex>

Without that, a 2GB droplet will get into OOM territory after a few weeks of normal traffic. The floor is officially ~1GB RAM, but plan for 1.5GB minimum if ClickHouse is sharing the host with anything else. The feature ceiling is also real: no funnels, no session replay, no custom event properties beyond a single goal URL. That's not a bug — Plausible is deliberately a pageview counter with a clean UI. Marketers who need conversion path analysis will hit that wall within a week.

Umami

The lightest option in this list by a significant margin. The Node.js app image sits under 200MB pulled, and at idle with a Postgres 16 backend you're looking at roughly 256MB RSS. That makes it viable on the smallest cloud instances or alongside other services on a 1GB box. The v2.x release added custom event properties, which matters — v1.x custom events were just named pings with no payload, which is nearly useless for anything beyond "button clicked". Make sure you're actually on v2 before assuming properties work.

The silent failure mode that will waste your afternoon: the DATABASE_URL environment variable must be properly URL-encoded. If your Postgres password contains @, #, or !, the container starts, logs nothing obviously wrong, and then fails auth on every request. Encode the password component with encodeURIComponent() in Node or just use a password without special characters in dev. The error surface is bad enough that it reads like a network issue, not a config issue.

# Wrong — will silently fail auth if password contains special chars
DATABASE_URL=postgresql://umami:p@ss#word@localhost:5432/umami

# Correct
DATABASE_URL=postgresql://umami:p%40ss%23word@localhost:5432/umami

Matomo

The most feature-complete tool here, and it earns that reputation: funnels, heatmaps, A/B testing, and GDPR consent tooling are all included without a SaaS upsell. The catch is operational complexity. The recommended split-container setup — php-fpm for processing, Nginx as the reverse proxy, MySQL or MariaDB for storage — has more moving parts than the others, and the Nginx config needs to correctly proxy to the PHP socket or you get a blank screen with no useful error.

The single most common operational failure on Matomo is skipping the archiving cron. Reports don't generate in real-time — they're computed from raw logs on a schedule. If you don't set this up, your dashboard freezes at the last archived date and looks like tracking broke:

# Add to crontab — run as the web server user, not root
*/5 * * * * /usr/bin/php /var/www/html/console core:archive --url=https://your-matomo-domain.com > /dev/null 2>&1

The --url flag is required and must match the Matomo install URL exactly, including protocol. Get it wrong and the archiver exits silently. Matomo is the right call when you need a full analytics suite that a non-technical marketer can use without training — it's the closest self-hosted equivalent to Google Analytics in terms of UI familiarity.

PostHog (Open-Source)

The docker-compose.hobby.yml that PostHog ships for self-hosters is not a lightweight deployment. It pulls in ClickHouse, Kafka, Redis, a plugin server, a Celery worker, and the main Django app. On a fresh 4GB RAM machine, the stack consumes roughly 3–3.5GB at idle before you've tracked a single event. That's not a complaint — PostHog is solving a fundamentally different problem than pageview counting. Feature flags, session replay, funnel analysis, and cohort tracking in one open-source stack is genuinely remarkable. But running it on a $6/month VPS to replace Google Analytics for a brochure site is the wrong trade-off. Use PostHog when you're tracking logged-in user behavior in a SaaS product and need the full product analytics loop. Use anything else on this list for marketing site traffic.

Ackee

Ackee's privacy model is different from the others: no cookies, no fingerprinting, and unique visitor counts are derived by hashing IP + User-Agent once per day and discarding the raw values. You cannot reconstruct individual sessions from what Ackee stores. The architecture is Node.js talking to MongoDB, and the footprint is minimal. The sharp edge: Ackee doesn't have a marketer-facing dashboard worth showing to a client. The UI is sparse, and the primary interface for anything beyond the basic view is a GraphQL API. That's actually the point — if you want to pipe raw visit counts into your own reporting layer, Notion embed, or n8n workflow, the GraphQL endpoint is clean and easy to work with. If a non-technical user needs to check traffic independently, send them to Plausible or Umami instead.

Deployment Configs Worth Copying

The most expensive lesson with self-hosted analytics isn't picking the wrong tool — it's losing months of data because a named volume wasn't declared before running docker compose down. Every config below is structured around the failure modes I've actually hit, not the happy-path examples in the official docs.

Plausible

Plausible's compose setup has two non-negotiable variables. BASE_URL must be the exact public URL your tracking script will report to — get this wrong and events silently 404. SECRET_KEY_BASE must be a real random value, not a placeholder:

# generate once, paste into .env, never rotate without migrating sessions
openssl rand -base64 64

version: "3.8"
services:
  plausible:
    image: ghcr.io/plausible/community-edition:v2.1.0
    env_file: .env
    depends_on:
      - db
      - clickhouse
    ports:
      - "8000:8000"

  clickhouse:
    image: clickhouse/clickhouse-server:23.3-alpine
    volumes:
      - clickhouse_data:/var/lib/clickhouse  # omit this and a compose down wipes all event history

volumes:
  clickhouse_data:  # named volume — required, not optional

After you've created the first admin account, add DISABLE_REGISTRATION=true to your .env and restart. Without it, anyone who finds your hostname can register. There's no rate limiting on the signup endpoint.

Umami

Umami's silent failure mode: if you skip the Prisma migration on first boot, the database schema is never initialized, but the container starts cleanly with no errors in logs. The UI just… doesn't work, and you'll spend time checking DNS before realizing the schema is empty.

# Run this once after first boot — not in an entrypoint, do it manually
docker compose exec umami npx prisma migrate deploy

# Minimal .env for Umami
DATABASE_URL=postgresql://umami:password@db:5432/umami
HASH_SALT=any-long-random-string-you-generate  # salts session fingerprints for privacy

HASH_SALT isn't just a security formality — it's what prevents Umami from storing raw IP-derived identifiers. Change it after data is collected and all historical session groupings break. Set it once and treat it like a private key.

Matomo

Matomo's archiving step is the single most common reason people think their tracking is broken when it isn't. Raw visits land in the database fine, but the report UI queries pre-aggregated archive tables. Without a cron job running the archiver, every date range shows "No data for this period" — even while events are being recorded in real time.

version: "3.8"
services:
  matomo:
    image: matomo:5.0-apache

  cron:
    image: matomo:5.0-apache
    # same image, different entrypoint — shares the same mounted config
    volumes:
      - matomo_data:/var/www/html
    entrypoint: /bin/sh -c "echo '0 * * * * www-data php /var/www/html/console core:archive --url=https://your-domain.tld >> /var/log/matomo-archive.log 2>&1' | crontab - && cron -f"

The --url flag must match the Matomo general settings URL exactly, including scheme. A mismatch causes the archiver to authenticate against the wrong host and exit silently. Pipe output to a log file you actually check — the default is to discard it.

PostHog

PostHog's hobby compose stack bundles MinIO for object storage because session recordings need somewhere to put binary blobs. The gotcha: if you set OBJECT_STORAGE_ENABLED=false to reclaim the ~300–400 MB of RAM MinIO holds, session recordings don't throw an error. They just silently drop. The UI shows the recording list, users appear in it, but playback is a spinner.

# Before assuming recordings work, check the worker logs
docker compose logs worker --tail=100 | grep -i "recording\|object_storage\|minio"

If you're RAM-constrained enough to disable MinIO, turn off session recording entirely in PostHog's project settings — that way you're intentionally missing data, not accidentally missing it. The worker log is the canonical truth here; the frontend gives you no signal either way.

Comparison Table: What Matters at the Operator Level

The gap between "it boots" and "it runs reliably for six months" is where most self-hosted analytics projects fall apart. These numbers and gotchas come from documented behavior, official resource requirements, and the kinds of failure modes that show up in GitHub issues after week two of production use.

Memory Footprint at Idle

Idle RAM is what determines whether this fits on a $6/month VPS or needs its own box. Ackee is the clear winner here — roughly 128MB at idle, because it's a Node process backed by MongoDB with almost no in-process caching. Umami sits around 256MB idle, which is reasonable for a Next.js app with a Postgres connection pool. Matomo at ~512MB is acceptable if you're already running a LAMP-adjacent stack. Then the cliff: Plausible's ClickHouse requirement means you're committing roughly 1GB before a single pageview lands, because ClickHouse doesn't release its memory reservation between queries. PostHog at ~4GB idle isn't a typo — that's Postgres, ClickHouse, Kafka, Redis, and the Django app server all running simultaneously. Deploying PostHog on anything under 8GB RAM is an exercise in OOM frustration.

# PostHog's own docker-compose uses these service minimums:
# postgres:      512MB
# clickhouse:    1GB+
# kafka:         512MB
# redis:         128MB
# web/worker:    1-2GB
# Total floor:   ~4GB — and that's before actual traffic load

Database Backends and What They Actually Mean

Every extra database in the stack is another thing to back up, tune, and potentially watch die at 2am. Ackee's MongoDB dependency is its most controversial design choice — you're adding a document store just to track page views, which feels heavy for what it does. Umami's Postgres-or-MySQL flexibility is genuinely useful; if you already have a managed Postgres instance, Umami slots in cleanly. Plausible's dual-database architecture (Postgres for account data, ClickHouse for event storage) is the right call at scale, but ClickHouse has a known memory creep behavior: without explicit max_memory_usage and max_memory_usage_for_user limits set in clickhouse-server/config.d/, it will gradually consume available RAM over days or weeks, especially with frequent aggregation queries. PostHog adds Kafka to that Postgres + ClickHouse combination, which means you now have a message queue to manage — useful for buffering ingestion spikes, but a source of delayed data when Kafka consumer lag builds up under traffic bursts.

# Plausible ClickHouse memory cap — add to your config.d/override.xml:

2000000000 
1500000000

The Gotcha That Will Actually Burn You

Umami's silent auth failure on a malformed database URL is probably the most dangerous because it's invisible — the container starts, health checks pass, but no events are being stored. The log output doesn't always surface a clear connection error; you find out when your dashboard shows a flatline two days later. Matomo's stale reports issue is well-known but still catches people: without the archiving cron running on schedule, the UI will serve cached aggregate data that's hours or days old while appearing current. The fix is straightforward but not obvious from the default install docs:

# Matomo archiving cron — add to crontab on the host running PHP
*/5 * * * * www-data /usr/bin/php /var/www/html/matomo/console core:archive \
  --url=https://your-matomo-domain.com > /dev/null 2>&1

PostHog's Kafka lag under traffic spikes is an operational reality rather than a bug. Events get buffered in Kafka, the consumer workers process them asynchronously, and during a spike you can have a 10–30 minute delay between a user action and that event appearing in the PostHog UI. For most small business use cases this is fine — but if you're watching a product launch in real-time, it will feel broken when it isn't.

Multi-Site Support: Free vs. Gated

All five tools support tracking multiple domains, but the licensing situation on Matomo deserves a callout. The multi-site feature is completely free and well-implemented in self-hosted Matomo — you manage a global view across all properties from one interface. The cloud version charges for it. If you're evaluating Matomo cloud pricing and wondering why multi-site is a paid add-on while the self-hosted docs treat it as a default feature, that's intentional product segmentation. PostHog's approach is structurally different: sites are "projects" under an organization, each with their own API key and isolated event stream, but sharing a single Kafka/ClickHouse backend. That means one PostHog deployment handles multi-tenant traffic efficiently, but schema changes or ClickHouse maintenance affect all your projects simultaneously.

When to Pick Which Tool

The fastest way to make the wrong choice here is to pick the tool with the most GitHub stars or the prettiest landing page. The right call depends almost entirely on your existing infrastructure, your ops tolerance, and whether you're tracking sessions or events — those are genuinely different problems with different right answers.

Pick Umami if you run a personal site or small blog and already have a Postgres instance sitting around. The migration path is a single docker-compose up, the schema is dead simple, and the dashboard gives you what 90% of non-technical stakeholders actually want to see. I've had Umami running for months without touching it — no log rotation surprises, no memory creep, no ClickHouse vacuum jobs. It's the only tool on this list where "near-zero ops" is actually true rather than aspirational marketing copy.

Pick Plausible Community Edition if you're migrating a client or marketing team off GA4 and need the UI to sell itself without a 20-minute onboarding call. The catch nobody mentions upfront: ClickHouse is not optional, and if you skip the memory config before your first traffic spike, you will hit OOM kills. Add this to your ClickHouse config.xml from day one:

<profiles>
  <default>
    <max_memory_usage>1073741824</max_memory_usage>
  </default>
</profiles>

Budget 1–2GB RAM above what you'd normally allocate just for the ClickHouse process, and don't run this on a 1GB VPS. Plausible is polished enough that non-technical stakeholders will actually trust the numbers, which is worth the extra infra overhead in client-facing contexts.

Pick Matomo if your use case is GDPR compliance documentation, funnel analysis, or heatmaps — and you're genuinely comfortable running a PHP application backed by MySQL long-term. The feature depth is real: Matomo's funnel and goal tracking is legitimately GA360-tier, and the consent management tooling is the most defensible of anything on this list for regulated industries. But the ops surface is the highest here. You're maintaining PHP runtime versions, MySQL slow query logs, and a plugin ecosystem that varies wildly in quality. Go in with eyes open.

Pick PostHog if you need to track product behavior — button clicks, form completions, feature flag exposure, session replays — alongside pageviews. The pageview tracking is almost incidental to what PostHog is actually built for. Using it purely as a hit counter is genuinely wasteful: it requires at least 4GB RAM to run the full stack without constant swap pressure, and that overhead only makes sense when you're using the event pipeline, cohort analysis, or the feature flag system. Pick Ackee at the opposite end: no sessions, no funnels, no retention graphs — just a clean GraphQL API that returns raw aggregated numbers. If you're piping analytics data into a custom dashboard or an n8n workflow that does its own aggregation, Ackee's minimalism is a deliberate architectural choice, not a missing feature. On my n8n setup I can query Ackee's API directly in an HTTP Request node and reshape the data however the downstream step needs it, without fighting an opinionated data model.

Keeping These Running: Backup and Upgrade Notes

The ClickHouse gotcha will ruin your week if you're not ready for it. Plausible and PostHog both use ClickHouse under the hood, and ClickHouse does not support naive major-version upgrades — pulling a new image tag and running docker compose up will either refuse to start or silently corrupt your data depending on the version gap. The fix is simple but easy to skip: pin your image tags explicitly in your compose file and read the ClickHouse changelog before you touch anything. A latest tag in production is how you find this out the hard way.

# docker-compose.yml — pin everything, never use :latest for stateful services
services:
  clickhouse:
    image: clickhouse/clickhouse-server:24.3.3.102  # pin to exact patch
    # before upgrading: read https://clickhouse.com/docs/en/whats-new/changelog
    # major version jumps require running the migration step first:
    # docker exec -it clickhouse clickhouse-client --query "SELECT version()"
    # then follow the upgrade guide for your specific version delta
  plausible:
    image: ghcr.io/plausible/community-edition:v2.1.1  # match to ClickHouse compat matrix

Matomo's backup requirements are slightly non-obvious. The MySQL database is the obvious thing to dump, but the file that actually controls your instance state is config/config.ini.php. That file holds your salted password hash configuration, your plugin enable/disable state, and your database credentials. If you restore the DB without it, Matomo either refuses to run or walks you through a fresh install as if no data exists. Back up both, keep them together, and test the restore. A minimal working backup script:

#!/bin/bash
# matomo-backup.sh — run nightly via cron
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/opt/backups/matomo"

# DB dump
mysqldump -u matomo_user -p"$MATOMO_DB_PASS" matomo_db | \
  gzip > "$BACKUP_DIR/db_$TIMESTAMP.sql.gz"

# Config file — small but critical
cp /opt/matomo/config/config.ini.php \
  "$BACKUP_DIR/config_$TIMESTAMP.ini.php"

# Prune backups older than 14 days
find "$BACKUP_DIR" -mtime +14 -delete

Umami's backup story is simpler because it's pure Postgres — no auxiliary config files, no proprietary state. A nightly pg_dump piped to Backblaze B2 via rclone covers the entire restore path. Backblaze B2's S3-compatible API means no custom tooling required, and the restore is a single psql command. On my setup this runs as a cron job on the host, not inside the container, so a crashed container doesn't also kill the backup process.

#!/bin/bash
# umami-backup.sh
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
DUMP_FILE="/tmp/umami_$TIMESTAMP.sql.gz"

# pg_dump directly from host; adjust connection string to match your compose env
PGPASSWORD="$UMAMI_DB_PASS" pg_dump \
  -h localhost -p 5432 -U umami_user umami_db | \
  gzip > "$DUMP_FILE"

# rclone must be configured with your B2 credentials: rclone config
rclone copy "$DUMP_FILE" b2:your-bucket-name/umami/
rm "$DUMP_FILE"

Every tool covered here exposes some form of health endpoint — Plausible at /api/health, PostHog at /health, Umami at /api/health, Matomo via its status page. Wire all of them to Uptime Kuma, which runs cleanly in the same Docker network and can hit internal hostnames directly without exposing anything to the internet. The one alert that actually matters beyond basic uptime: Matomo's archiving cron. When it fails, the dashboard shows stale aggregated data with no visible error — users just notice that today's numbers look wrong. Add a cron job that writes a heartbeat file on success, and monitor that file's mtime from Uptime Kuma using its "Keyword" check type or a simple shell monitor.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

Self-Hosted Analytics Without Google: Plausible, Umami, and Matomo on Your Own Hardware

우병수 — Wed, 05 Aug 2026 08:11:28 +0000

TL;DR: GA4's event model is the first thing that breaks trust. The old Universal Analytics had one job: tell you how many people read a page.

📖 Reading time: ~22 min

What's in this article

Why Self-Hosted Operators Stop Trusting Google Analytics
The Three Contenders: What Each Tool Actually Is
Comparison Table: Hardware Requirements and Operational Reality
Setup to Production: Docker Compose Configs for Each
Non-Obvious Behaviors That Cost Time in Production
When to Pick What: Matching the Tool to the Actual Situation
Ongoing Ops: Backups, Updates, and Staying Out of Trouble

Why Self-Hosted Operators Stop Trusting Google Analytics

GA4's event model is the first thing that breaks trust. The old Universal Analytics had one job: tell you how many people read a page. GA4 replaced that with a flexible event schema that's genuinely powerful for e-commerce funnels — and genuinely annoying if you just want a pageview count you can trust. Getting that number now means navigating the Explorations interface, picking the right event, filtering by page_location, and hoping you didn't hit the sampling threshold that kicks in on date ranges longer than a few weeks. A self-hosted blog operator shouldn't need to reverse-engineer a BI tool to answer "did anyone read Tuesday's post?"

The data residency problem is more fundamental than the UX complaints. If you're already running your own Nginx, your own Postgres, your own monitoring stack — routing your visitors' behavior through Google's infrastructure is architecturally incoherent. It's not paranoia; it's consistency. GDPR makes this concrete: any EU visitor whose data hits Google's servers requires a consent mechanism, which in practice means a cookie banner, a Consent Management Platform, or both. The CMP itself adds another 20–80 KB of JavaScript depending on vendor, it fires before your content loads, and it tanks your Lighthouse scores on mobile. You're paying a performance penalty to collect data you don't own.

The real cost accounting looks like this: the GA4 loader script ships around 45 KB minified (closer to 17 KB gzipped, but it also triggers additional async fetches). That's not catastrophic in isolation, but stack it against a consent banner blocking render, the CMP payload, and the fact that GA4 stores aggregated data with no raw event export on the free tier, and the "free" label stops being accurate. You don't get a SQL table. You get a dashboard that Google controls, with sampling on queries that touch more than a few months of history, and no way to backfill if you change your event schema. Compare that to running Plausible or Umami in Docker: a single container, raw data in Postgres or ClickHouse, zero third-party calls, and no consent banner required under most GDPR guidance because no personal data leaves your server.

The build-vs-rent tension here is identical to what comes up with AI tooling — you can rent Google's analytics infrastructure the same way you can rent a cloud copilot, and both choices come with the same hidden costs: rate limits you don't control, data you can't fully export, and a pricing/feature surface that can change under you. For operators already thinking through that tradeoff in other contexts, see our breakdown of AI Coding Tools in 2026: Cloud Copilots vs Local Models — the reasoning transfers directly. Once you've run your own LLM inference or your own monitoring stack, the question isn't whether to self-host analytics; it's which tool fits your traffic volume and query patterns.

The Three Contenders: What Each Tool Actually Is

Plausible v2.x surprises most people with its stack choice: it's written in Go with an Elixir layer (the Elixir/Phoenix frontend, Go for the ingestion path), backed by ClickHouse for event storage rather than PostgreSQL — which is where its query speed comes from on high-traffic sites. The ~1 KB script isn't marketing copy; load it and inspect it yourself in DevTools. There's no cookie set, no fingerprinting, no personal data written to disk. That's not just a privacy stance — it means you genuinely don't need a consent banner under GDPR or ePrivacy, which removes an entire category of compliance work. The trade-off is that Plausible's data model is intentionally coarse: sessions are approximated, not tracked across requests, so you won't get per-user journey data. That's a feature if you don't want it, a blocker if you do.

Umami v2.x runs on Next.js 14+ with either PostgreSQL 15+ or MySQL 8+, and it's the most developer-friendly of the three to modify. The dashboard is a React app you can fork, restyle, or embed. Event tracking is a dead-simple API call:

// Send a custom event from any JS context
fetch('https://your-umami-host/api/send', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    payload: {
      hostname: window.location.hostname,
      language: navigator.language,
      referrer: document.referrer,
      screen: `${window.screen.width}x${window.screen.height}`,
      title: "document.title,"
      url: window.location.pathname,
      website: 'YOUR_WEBSITE_UUID',
      name: 'button_click',      // event name
      data: { plan: 'pro' }      // arbitrary JSON payload
    },
    type: 'event'
  })
});

Cookies are off by default in recent versions, though you can enable session persistence if you need it. The honest limitation: Umami's query layer is straightforward SQL — fine for most sites, but if you're pushing millions of events per day, you'll feel the difference versus ClickHouse-backed tools. It also doesn't ship with funnel analysis, heatmaps, or e-commerce tracking out of the box.

Matomo v5.x is the only one of these three that can genuinely replace Google Analytics feature-for-feature. PHP 8.2+ and MySQL 8+ (or MariaDB 10.6+) are the current requirements. You get funnels, goal tracking, e-commerce revenue attribution, campaign tagging, and raw SQL access to your own database — no data locked in a vendor's warehouse. Heatmaps and session recordings exist but are paid plugins through the Matomo Marketplace, which is a real cost to factor in. The operational weight is also real: Matomo runs a cron job for log processing, the archive process hammers your database on high-traffic sites, and the plugin ecosystem means you're managing PHP dependencies. Expect to allocate at least 2 GB RAM to a Matomo instance under any meaningful load, versus Plausible and Umami both running comfortably under 512 MB for most self-hosted scenarios.

The practical split: Plausible if you want the minimum viable analytics footprint with zero compliance friction. Umami if you're already in a Next.js ecosystem or want to customize the tool itself. Matomo if you're replacing GA3 and need feature parity — funnels, e-commerce, or a client who expects a specific report that the lighter tools simply can't produce.

Comparison Table: Hardware Requirements and Operational Reality

Hardware Requirements and Operational Reality

The gap between "runs on my machine" and "runs reliably at 3am when nobody's watching" is mostly a resource-sizing problem. These three tools have genuinely different operational profiles — not just different RAM numbers, but different failure modes when those numbers get tight. The worst surprises come from components that aren't the main application: ClickHouse for Plausible, the archiving queue for Matomo, MySQL tuning for Umami under write pressure.

Dimension

Plausible CE

Umami

Matomo

Minimum RAM (container stack)

~512 MB app + ~1 GB ClickHouse = ~1.5 GB realistic floor

~256 MB app + whatever your existing Postgres/MySQL uses

~1 GB PHP-FPM + MySQL under any real traffic

Idle disk write rate

ClickHouse merges parts in background; expect non-trivial I/O even at idle — not friendly to NVMe write-endurance budgets on small VPS

Low — standard DB writes, no background compaction engine; quiet on low-traffic sites

Archiving cron writes aggregates to MySQL; can spike I/O heavily during catch-up runs on busy sites

Tracker script payload

~1 KB

~2 KB

~22 KB default (configurable; a minimal build gets it lower, but requires deliberate effort)

Database engine support

ClickHouse only (no swapping it out)

PostgreSQL 12+ or MySQL 5.7+ — reuse existing infrastructure

MySQL / MariaDB (primary); experimental Postgres support exists but isn't production-recommended

Single biggest operational dealbreaker

ClickHouse overhead on a low-RAM VPS. On a 1 GB or 2 GB node, ClickHouse competes with the Plausible app itself and will OOM under aggregation pressure. The community edition moved aggregations to ClickHouse — you can't opt out.

Reporting depth. Umami gives you pageviews, referrers, devices, and custom events — but no funnel analysis, no goal tracking UI, no segmentation engine. If you outgrow the basics, you're either querying the DB directly or moving tools.

The archiving cron job. Matomo doesn't aggregate in real-time — it batches via core:archive. If the cron falls behind (high traffic, slow disk, misconfigured PHP memory limit), your dashboards show stale data and the unprocessed queue grows until you manually intervene or the next cron window opens.

The archiving problem with Matomo deserves more attention than it usually gets. The cron command is:

# run as www-data or the PHP user, not root
/usr/bin/php /var/www/matomo/console core:archive \
  --url=https://your-matomo-instance.example.com \
  --php-cli-options="-d memory_limit=2048M"

If memory_limit is too low in the CLI context (separate from the web php.ini), the archiver silently exits mid-run. Dashboards look fine until you notice the "last processed" timestamp hasn't moved in six hours. The fix is trivial once you know it, but the failure is quiet — no alert, no error page, just stale numbers. Set up an external check on that timestamp or you will miss it.

Umami's low RAM floor is the real differentiator for shared or budget hardware. If you're already running Postgres 15 or MySQL 8 for something else on the same host, Umami adds essentially nothing to your resource budget — just another schema in an existing instance. Plausible's ClickHouse dependency is non-negotiable in the current self-hosted CE, and ClickHouse behaves badly under memory pressure in ways that are hard to tune without deep ClickHouse knowledge. For anything under 4 GB total host RAM, Plausible CE becomes a risky choice unless you dedicate the box to it.

Setup to Production: Docker Compose Configs for Each

The gap between "it runs" and "it runs correctly" is almost entirely in the config details the official docs bury or skip. Here's what each stack actually needs.

Plausible CE: Three Services, One Footgun

The official docker-compose.yml spins up three containers — plausible, plausible_db (Postgres 16), and clickhouse — and the ClickHouse one will silently OOM on a 1 GB VPS if you don't constrain it. The fix lives in a mounted XML config, not in compose env vars.

# clickhouse-config.xml — mount this into the clickhouse container
<?xml version="1.0"?>
<clickhouse>
  <max_server_memory_usage_to_ram_ratio>0.4</max_server_memory_usage_to_ram_ratio>
</clickhouse>

# docker-compose.yml (Plausible CE — trimmed to the non-obvious bits)
version: "3.8"
services:
  plausible:
    image: ghcr.io/plausible/community-edition:v2.1.1
    restart: unless-stopped
    depends_on:
      - plausible_db
      - clickhouse
    environment:
      BASE_URL: "https://stats.yourdomain.com"
      SECRET_KEY_BASE: "REPLACE_WITH_64_CHAR_HEX"   # openssl rand -hex 64
      TOTP_VAULT_KEY: "REPLACE_WITH_BASE64_KEY"      # openssl rand -base64 32
    ports:
      - "127.0.0.1:8000:8000"

  plausible_db:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: plausible_db
      POSTGRES_USER: plausible
      POSTGRES_PASSWORD: REPLACE_ME
    volumes:
      - pg_data:/var/lib/postgresql/data

  clickhouse:
    image: clickhouse/clickhouse-server:24.3-alpine
    restart: unless-stopped
    volumes:
      - ./clickhouse-config.xml:/etc/clickhouse-server/config.d/memory.xml:ro
      - ch_data:/var/lib/clickhouse

volumes:
  pg_data:
  ch_data:

SECRET_KEY_BASE and TOTP_VAULT_KEY are both required on first run — omitting TOTP_VAULT_KEY doesn't throw an obvious error, it just breaks TOTP enrollment silently. Generate them before you start the stack, not after. The 0.4 ratio on ClickHouse caps it at roughly 400 MB on a 1 GB host; tune it up on bigger boxes but never skip it.

Umami: The Lean Two-Service Option

Umami's compose is genuinely simple — one app container, one database. The entire database config lives in a single DATABASE_URL env var, which makes environment swaps trivial. It works with either Postgres or MySQL; Postgres is the better call if you're already running it for other services.

version: "3.8"
services:
  umami:
    image: ghcr.io/umami-software/umami:postgresql-v2.13.2
    restart: unless-stopped
    depends_on:
      - umami_db
    environment:
      DATABASE_URL: "postgresql://umami:REPLACE_ME@umami_db:5432/umami"
      APP_SECRET: "REPLACE_WITH_RANDOM_STRING"
    ports:
      - "127.0.0.1:3000:3000"

  umami_db:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: umami
      POSTGRES_USER: umami
      POSTGRES_PASSWORD: REPLACE_ME
    volumes:
      - umami_pg:/var/lib/postgresql/data

volumes:
  umami_pg:

The tracking script endpoint is /script.js and the event collection endpoint is /api/send. That second one is useful: you can POST custom events directly from an n8n HTTP Request node without touching the browser SDK at all. The payload is straightforward JSON — {"type":"event","payload":{"website":"YOUR_WEBSITE_ID","url":"/","name":"custom_event"}} — which means any workflow that reaches an HTTP node can push telemetry into Umami. On my n8n flows I use this to track non-browser actions like newsletter sends or RSS fetch completions.

Matomo: The Cron Container You Can't Skip

Matomo's official images are matomo:5-apache (simpler) and matomo:5-fpm (pairs with a separate nginx container, slightly lower memory per request). The less-documented requirement is the archiver — without it, your reports are raw unprocessed logs and the UI shows stale or empty data. The exact command is:

# Run this every hour via cron or a dedicated compose service
php /var/www/html/console core:archive --url=https://your-matomo-domain

version: "3.8"
services:
  matomo:
    image: matomo:5-apache
    restart: unless-stopped
    depends_on:
      - matomo_db
    environment:
      MATOMO_DATABASE_HOST: matomo_db
      MATOMO_DATABASE_DBNAME: matomo
      MATOMO_DATABASE_USERNAME: matomo
      MATOMO_DATABASE_PASSWORD: REPLACE_ME
    volumes:
      - matomo_html:/var/www/html
    ports:
      - "127.0.0.1:8080:80"

  matomo_cron:
    image: matomo:5-apache
    restart: unless-stopped
    depends_on:
      - matomo
    volumes:
      - matomo_html:/var/www/html   # shared volume so cron sees same config
    entrypoint: |
      sh -c "while true; do
        php /var/www/html/console core:archive --url=https://your-matomo-domain;
        sleep 3600;
      done"

  matomo_db:
    image: mariadb:11.4
    restart: unless-stopped
    environment:
      MYSQL_DATABASE: matomo
      MYSQL_USER: matomo
      MYSQL_PASSWORD: REPLACE_ME
      MYSQL_ROOT_PASSWORD: REPLACE_ROOT_ME
    volumes:
      - matomo_db_data:/var/lib/mysql

volumes:
  matomo_html:
  matomo_db_data:

The shared matomo_html volume between the main container and the cron container is the key detail here — the archiver needs access to config/config.ini.php which is written on first-run setup. If the cron container mounts a separate or empty volume, it finds no config and exits silently with a non-zero code that's easy to miss. MariaDB 11.4 is the current LTS and what Matomo's installer validates against; Postgres support in Matomo is officially experimental as of v5.

Reverse Proxy: Where Each Tool Has Its Own Quirk

All three sit cleanly behind Nginx or Caddy on 127.0.0.1 ports, but each has a config requirement that trips people up in production. Plausible needs accurate X-Forwarded-For headers or every visitor logs as your proxy's IP. Matomo has the same problem but fixes it in PHP config rather than relying purely on the proxy headers.

# Nginx — Plausible block (X-Forwarded-For is the critical line)
location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Host $host;
}

# Matomo: config/config.ini.php — add under [General]
# Without this, Matomo logs 127.0.0.1 as every visitor's IP
[General]
trusted_proxies[] = "127.0.0.1"
proxy_client_headers[] = "HTTP_X_FORWARDED_FOR"
proxy_host_headers[] = "HTTP_X_FORWARDED_HOST"

Caddy handles X-Forwarded-For automatically when you use the reverse_proxy directive — it's one of the reasons I prefer Caddy for new self-hosted stacks. With Nginx you have to be explicit every time. For Umami there's no special proxy config required; it reads client IPs from the forwarded headers without extra configuration as long as the header reaches the app container. One Nginx-specific gotcha with Plausible: if you're behind a CDN that also adds X-Forwarded-For, the header can contain a comma-separated chain of IPs — Plausible reads the leftmost one, which is correct behavior, but confirm your CDN isn't mangling the header order before blaming geolocation accuracy.

Non-Obvious Behaviors That Cost Time in Production

The one that catches the most operators off guard with Plausible: ClickHouse will silently accept write requests and return HTTP 200s while Plausible is mid-write — but if the ClickHouse container restarts at that moment, you end up with an inconsistent state where Plausible's dashboard returns empty charts for that period. No error in the Plausible logs. No obvious failure. Just a gap. The fix is straightforward but non-obvious if you haven't been burned by it: set restart: always on both the Plausible and ClickHouse containers in your Compose file, and add a real health check to ClickHouse that uses clickhouse-client rather than a generic TCP ping:

services:
  clickhouse:
    image: clickhouse/clickhouse-server:24.3
    restart: always
    healthcheck:
      test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

  plausible:
    image: ghcr.io/plausible/community-edition:v2.1.4
    restart: always
    depends_on:
      clickhouse:
        condition: service_healthy

The depends_on with condition: service_healthy is the part most Docker Compose tutorials skip. Without it, Plausible starts, tries to connect before ClickHouse is actually ready to serve queries, and you get a subtler version of the same gap problem — especially after host reboots.

Umami's event tracking API is deceptively permissive. You can POST arbitrary JSON to the /api/send endpoint and the server will accept it without complaint. The trap is that the default Umami dashboard UI only surfaces string and number property values — anything else either gets coerced silently or ignored at render time. If you're trying to replicate GA's custom dimensions behavior and you want to filter or segment by those properties, you'll hit a wall in the UI almost immediately. The actual data is in the PostgreSQL website_event and event_data tables, and querying it directly is the only real path to GA-style segmentation. A query that surfaces custom string properties for a given event name looks like this:

SELECT
  we.created_at,
  we.event_name,
  ed.string_value,
  ed.number_value
FROM website_event we
JOIN event_data ed ON ed.website_event_id = we.id
WHERE we.website_id = 'your-website-uuid'
  AND we.event_name = 'purchase'
ORDER BY we.created_at DESC
LIMIT 100;

Matomo's archiving cron is the most production-hostile behavior of the three. The archiving process is stateful — Matomo tracks which periods have been archived in the database, and if a cron run fails partway through (the most common cause being PHP hitting its default memory_limit of 128M on sites with any real traffic), the next scheduled run re-attempts the same period from scratch. Two back-to-back failures on a busy day means the third run tries to process three periods simultaneously. The load compounds fast. The fix requires two things: bump memory_limit = 256M in your php.ini (512M if you have months of historical data being re-archived), and route cron output to a file you can actually grep:

# in crontab -e
*/15 * * * * www-data /usr/bin/php /var/www/matomo/console core:archive \
  --url=https://matomo.yourdomain.com \
  >> /var/log/matomo/archive.log 2>&1

Without that log file, you're flying blind. Matomo's UI will show "archiving in progress" indefinitely if the cron process dies mid-run, and there's no dashboard alert for it. Tail that log during the first week after any Matomo upgrade — the memory pressure almost always spikes after schema migrations.

All three tools share one unavoidable problem: Firefox with Enhanced Tracking Protection enabled and Brave with default shield settings will block the tracking scripts before they reach the browser. This isn't a minor edge case — on a developer or privacy-conscious audience, a meaningful chunk of your actual visitors simply won't register. Plausible's official docs include a working Nginx proxy snippet that routes the script and event endpoint through your own domain:

location = /js/script.js {
    proxy_pass https://plausible.io/js/script.js;
    proxy_set_header Host plausible.io;
}

location = /api/event {
    proxy_pass https://plausible.io/api/event;
    proxy_set_header Host plausible.io;
    proxy_buffering on;
    proxy_http_version 1.1;
}

Umami sidesteps the proxy complexity entirely if you're self-hosting: since Umami runs on your own infrastructure, you can mount it at a path on your primary domain (e.g., yourdomain.com/stats/) and serve the tracking script from there. Browsers treating first-party paths as tracking domains are rare enough that this approach works in practice. Matomo also supports first-party cookie mode and a self-hosted script path, but the configuration is buried in the Tag Manager settings and the documentation assumes you already know where to look.

When to Pick What: Matching the Tool to the Actual Situation

The actual decision between these tools comes down to operational cost and data model requirements, not feature checklists. Running any of these for a few days reveals the real constraints faster than any comparison table.

Pick Plausible when you want the shortest path from "deployed" to "trustworthy data" — no cookie banner, no consent dialog, no GDPR checkbox engineering. The ClickHouse container is the honest constraint: expect the pair (Plausible + ClickHouse) to sit comfortably inside 2 GB RAM once warmed up, but plan for spiky memory during ClickHouse compaction if you're on a shared VPS. Once stable, it's the least operationally demanding of the three — ClickHouse rarely needs tuning at typical blog-scale traffic and the Plausible UI has no configuration surface to break. If your audience is technical or privacy-aware enough that they run uBlock or Brave, you're also going to see fewer gaps in your data than with anything cookie-dependent.

Pick Umami if you already have a running Postgres 14+ or MySQL 8 instance and don't want to add another database engine to your stack. The setup is genuinely fast — single Docker image, one DATABASE_URL env var, done. The bigger reason to pick Umami over Plausible is the /api/send endpoint: it accepts a plain POST with a JSON payload, which means your n8n flows or any server-side automation can push custom events without a browser involved. The footgun is real though — by default that endpoint has no authentication, so anyone who finds your tracker URL can inject events. Set DISABLE_TELEMETRY=1 and put a reverse-proxy auth layer in front before you expose this publicly:

# nginx snippet — drop requests to /api/send that don't carry your shared secret
location /api/send {
    # only allow your own automation; block browsers you don't control
    if ($http_x_tracker_secret != "your-secret-here") {
        return 403;
    }
    proxy_pass http://umami:3000;
}

Pick Matomo when the data you need simply doesn't fit into a page-view counter — goal funnels, e-commerce revenue rows, session recordings, or raw SQL access to the log_visit table for custom reporting. The honest trade-off: you're not running a container, you're running a PHP application with a MySQL/MariaDB backend, a cron job for archiving, and a plugin ecosystem that can drift. Migrating from Universal Analytics is the strongest case for Matomo — the goal and funnel data model maps closely enough that you don't have to redesign your measurement strategy from scratch. Accept the operational weight and you get the most complete local data store of the three.

Skip all three and route to GoAccess on your Nginx access logs if your situation is any of: you have zero JavaScript budget (AMP pages, email-linked landing pages, API-only services), you're trying to measure traffic that a JS snippet would miss entirely (RSS fetches, direct API consumers, bot traffic you actually want to count), or you just need referrer and page-view counts without standing up a database. GoAccess parses compressed log archives, outputs a self-contained HTML report, and adds nothing to your request path:

# generate a real-time HTML report from rotated Nginx logs
zcat /var/log/nginx/access.log.*.gz | \
  goacccess /var/log/nginx/access.log - \
  --log-format=COMBINED \
  --output=/var/www/html/stats/report.html

Run that in a PM2-managed cron and you have a zero-dependency analytics page that survives anything the other three tools won't — database crashes, container OOM kills, schema migrations that break on PHP version bumps. The data is coarser, but for a mostly-server-side or API workload it's more accurate than anything that relies on a browser firing a JavaScript beacon.

Ongoing Ops: Backups, Updates, and Staying Out of Trouble

The part that catches most people is assuming analytics data is recoverable after a bad upgrade. It usually isn't — not cleanly. By the time you notice the graphs stopped updating, the window for a clean rollback is already closed.

Plausible: Two Databases, Two Failure Modes

Plausible splits its state across Postgres and ClickHouse, and losing either one breaks the stack in completely different ways. Postgres holds user accounts, site configs, and API keys — lose it and the app won't boot cleanly. ClickHouse holds every pageview event — lose it and your dashboards are blank forever. Back up both, separately, on different schedules if you have to.

For Postgres, standard pg_dump is fine:

# Run from cron or an n8n schedule node
pg_dump -U plausible -d plausible_db | gzip > /backups/plausible-pg-$(date +%F).sql.gz

For ClickHouse, the cleanest option if you're running it in Docker is a volume snapshot at the host level, or clickhouse-backup if you want something scriptable:

# clickhouse-backup create — backs up to /var/lib/clickhouse/backup/
docker exec clickhouse clickhouse-backup create plausible-$(date +%F)
# Then rsync or rclone that directory off the host

The clickhouse-backup tool requires a config file at /etc/clickhouse-backup/config.yml inside the container. The docs skip that step and the binary just silently exits if the config is missing — check docker logs before trusting that the backup ran.

Umami: Simpler Stack, But Pin Your Image Tag

Umami is fully stateless on the app side — everything lives in Postgres, so a daily pg_dump is genuinely sufficient. What will burn you isn't the backup strategy, it's the image tag. The latest tag on ghcr.io/umami-software/umami has shipped breaking schema changes on minor bumps without warning. Pin to a specific release in your docker-compose.yml:

services:
  umami:
    image: ghcr.io/umami-software/umami:postgresql-v2.12.0
    # Do NOT use :latest in production
    restart: unless-stopped

When you want to upgrade, pull the changelog manually, bump the tag explicitly, and let Umami's built-in migration run on first boot with the new image. Rolling back means just reverting the tag and restoring the Postgres dump — the stateless app layer makes this fast.

Matomo: The Silent Migration Corruption Problem

Matomo is the most operationally fragile of the three on upgrades. Its database migration on version bumps touches archive tables, and if the migration fails partway through — which happens more often than the docs admit — it won't tell you loudly. The dashboards will load, the data will look present, and two weeks later you'll notice your historical reports are returning zeros or throwing SQL errors on aggregation queries.

Before any Matomo version upgrade, two things are non-negotiable. First, dump both the database and the config directory:

# MariaDB/MySQL dump
mysqldump -u matomo -p matomo_db | gzip > /backups/matomo-db-$(date +%F).sql.gz

# Config dir — holds local.php with DB credentials and installed plugin state
tar czf /backups/matomo-config-$(date +%F).tar.gz /var/www/html/config/

Second, hit the pre-update check before touching the image tag:

https://your-matomo-host/index.php?module=CoreUpdater

That endpoint will tell you if the pending migration has any prerequisites it can't meet. Skip it and you're gambling. The Matomo upgrade docs mention it in passing; treat it as mandatory.

All Three: You Need an Uptime Monitor on the Analytics Endpoint

Analytics data gaps are invisible until they're not. A container that OOMed at 3am, a ClickHouse writer that deadlocked and stopped accepting events, a Matomo cron that quietly stopped archiving — none of these will throw an obvious error. Your site keeps loading, users keep visiting, and the data just stops. You'll notice days later when a graph goes flat.

Uptime Kuma runs well in the same Docker network and adds negligible overhead. Point an HTTP monitor at each analytics ingestion endpoint, not just the dashboard UI:

Plausible: GET /api/event returns a 202 on a valid payload — monitor that, not the homepage
Umami: /api/send endpoint; a 400 on a malformed payload still confirms the app is alive
Matomo: /matomo.php?idsite=1&rec=1 — a 204 response means the tracker is accepting hits

Set the alert interval tight — five minutes is reasonable. A 24-hour gap in analytics data is already painful to explain; a week-long gap because nobody checked is a data loss event.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

5 Monitoring Tools That Actually Work for Self-Hosted Servers (Tested on Real Hardware)

우병수 — Mon, 03 Aug 2026 08:11:10 +0000

TL;DR: The alert that actually matters almost never fires. Instead, you find out a service is down because someone — or something — can't reach it and tells you directly.

📖 Reading time: ~20 min

What's in this article

The Monitoring Gap That Bites Self-Hosters
Tool 1: Prometheus + Node Exporter — The Baseline You Build Everything On
Tool 2: Grafana — Dashboards Worth Waking Up For
Tool 3: Netdata — When You Want Answers in 30 Seconds, Not 30 Minutes
Tool 4: Uptime Kuma — Endpoint Monitoring With Zero Complexity Tax
Tool 5: Zabbix — When the Complexity Is Actually Justified
Picking the Right Stack for Your Setup

The Monitoring Gap That Bites Self-Hosters

The alert that actually matters almost never fires. Instead, you find out a service is down because someone — or something — can't reach it and tells you directly. Your "monitoring" turns out to be a Grafana dashboard you check when you remember to, and a vague sense that things are probably fine. That gap between assumption and reality is where self-hosted setups get hurt: a model server OOM-killed at 3am, a disk that filled up because log rotation wasn't configured, a container that exited silently because a dependency socket disappeared. None of it paged you. You found out later.

What a single-node homelab or small self-hosted stack actually needs is different from what generic "monitoring" implies. You need process-level visibility — not just "is the host up" but "is ollama serve still running, and is it responding to inference requests." You need container metrics that show per-container CPU and memory churn, not just host-level aggregates. If you're running inference workloads, GPU and VRAM tracking is non-negotiable: a model that's been evicted from VRAM and is quietly swapping to CPU will still respond, just 10x slower, and nothing in basic monitoring will catch that unless you're watching nvidia-smi output or a metric derived from it. And you need alerting that fires before things break — disk at 85%, swap climbing, a container restart loop starting — not after the service is already gone.

Enterprise tools like Datadog and New Relic are designed around different assumptions entirely. Their agents are built for fleets — they expect many nodes, centralized collection infrastructure, and enough traffic to justify per-host or per-metric pricing. On a single 32GB workstation running Ollama, n8n, a Postgres instance, and a TypeScript publishing engine, the Datadog agent alone can consume a noticeable chunk of the RAM budget you were saving for a second model context. More critically, these platforms assume cloud connectivity as a baseline — their alerting pipelines, dashboards, and integrations all phone home. A local-first stack running airgapped or behind a NAT with no public endpoint doesn't fit the model they're optimized for. You end up paying for infrastructure assumptions that actively work against your setup.

The practical alternative is composing purpose-built open source tools: a lightweight metrics collector that exposes data locally, a time-series store you control, and alerting logic that sends to wherever you actually look — a Telegram bot, a webhook into your n8n flow, an email. The ecosystem for this has matured enough that you can get genuine process-level, container-level, and GPU-level observability running in under an hour, with no external dependencies and no per-host licensing math to do. If you are evaluating tooling across your stack, the guide on AI Coding Tools in 2026: Cloud Copilots vs Local Models covers the broader picture of what is worth running locally vs. delegating to a cloud API — the same decision framework applies here.

Tool 1: Prometheus + Node Exporter — The Baseline You Build Everything On

Most monitoring stacks get complicated fast, but Prometheus earns its place as a foundation because the mental model never changes: it polls HTTP endpoints that expose metrics in a plain-text format, timestamps them, and writes them into a local time-series database. No agent protocol to debug, no proprietary wire format, no daemon handshake. If curl http://localhost:9100/metrics returns data, Prometheus can scrape it. That simplicity is what makes it composable — Grafana, Alertmanager, and a dozen exporters all plug in without ceremony.

Getting it running takes maybe fifteen minutes. The compose file below is close to what I actually run, minus the Grafana container. The key decisions are bind-mounting your config (so you can edit it without rebuilding) and putting both containers on the same network so Prometheus can reach Node Exporter by service name:

# docker-compose.yml
services:
  prometheus:
    image: prom/prometheus:v2.52.0
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.retention.time=30d'
    ports:
      - "9090:9090"
    networks:
      - monitoring

  node_exporter:
    image: prom/node-exporter:v1.8.0
    pid: "host"           # needed for accurate process metrics
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.sysfs=/host/sys'
      - '--path.rootfs=/rootfs'
    ports:
      - "9100:9100"
    networks:
      - monitoring

volumes:
  prometheus_data:

networks:
  monitoring:

# prometheus.yml
global:
  scrape_interval: 5s        # override the 15s default — see note below
  evaluation_interval: 5s

scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets: ['node_exporter:9100']
        labels:
          host: 'my-workstation'   # shows up in every metric — label it now

The default 15-second scrape interval is the first thing worth changing. A process that allocates 20 GB of RAM and gets OOM-killed in under 10 seconds will leave zero evidence in your Prometheus data — the spike simply falls between samples. Dropping to 5 seconds on a single node adds negligible load; Node Exporter's /metrics endpoint is a cheap read from procfs, and Prometheus's own CPU overhead at that cadence on one target is unmeasurable against anything else running on the box. The tradeoff only becomes real when you're scraping dozens of targets or using expensive custom collectors.

Resource costs are lower than most people expect. Prometheus itself runs comfortably under 400 MB RSS on a single-node setup with 30-day retention and the 5-second interval. The disk math is straightforward: Prometheus compresses time-series data well, and at default Node Exporter cardinality you should expect roughly 1–2 GB per monitored host per 30 days. The number grows if you add high-cardinality exporters (cAdvisor with many containers, for instance), but for bare-metal hardware metrics alone, even a modest SSD partition handles years of retention. Run du -sh /var/lib/docker/volumes/prometheus_data after a week to calibrate your own rate before committing to a retention window.

Tool 2: Grafana — Dashboards Worth Waking Up For

Grafana as the Visualization Layer on Top of Prometheus

Grafana does not collect metrics itself — it reads from Prometheus (or Loki, or InfluxDB, but Prometheus is the workhorse here) and renders them. That separation is actually the right design: you get best-in-class storage and querying from Prometheus, and best-in-class visualization from Grafana, without either trying to do the other's job. The practical upside is that you can blow away your Grafana container entirely, redeploy it, reconnect it to the same Prometheus instance, and lose nothing except dashboards you forgot to back up.

Community Dashboards vs. Building Your Own

Start with dashboard ID 1860 — the "Node Exporter Full" dashboard available at grafana.com/dashboards. Import it via the UI (Dashboards → Import → enter 1860), point it at your Prometheus data source, and you immediately get CPU, memory, disk I/O, network throughput, and filesystem pressure across every host running node_exporter. That covers 80% of what you need for general server health without writing a single PromQL query. The remaining 20% — GPU VRAM on the Ollama box — requires building panels from scratch using metrics exposed by nvidia-smi through the NVIDIA DCGM exporter or the lighter nvidia_gpu_exporter. The metric you want is nvidia_smi_memory_used_bytes divided by nvidia_smi_memory_total_bytes, multiplied by 100, to get a clean VRAM utilization percentage. Wire that into a Gauge panel with thresholds at 75% (yellow) and 90% (red) and you have something actually useful for watching Ollama load a 13B model.

Docker Compose Addition

Drop this into the same docker-compose.yml that already runs your Prometheus container:

  grafana:
    image: grafana/grafana-oss:10.4.2
    container_name: grafana
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      # Set this BEFORE the container ever starts — Grafana writes the
      # admin password into grafana.db on first boot and ignores this
      # env var on subsequent starts unless you reset via CLI.
      GF_SECURITY_ADMIN_PASSWORD: "changeme_before_first_run"
      GF_USERS_ALLOW_SIGN_UP: "false"
    volumes:
      - grafana_data:/var/lib/grafana   # persists grafana.db, dashboards, alert rules

volumes:
  grafana_data:
    driver: local

The GF_SECURITY_ADMIN_PASSWORD timing matters more than the docs make clear. Grafana initializes grafana.db (a SQLite file inside the volume) on its very first startup, writing whatever password is set at that moment. If you start the container with the default password, log in, then try to change the env var and restart — nothing changes. The new value is silently ignored because the database already exists. The fix is to either set the password correctly before the first docker compose up, or run grafana-cli admin reset-admin-password inside the container after the fact. Either works; forgetting costs you time.

Alerting via Grafana Unified Alerting

Grafana v9 replaced the old alerting system with Unified Alerting, and it is meaningfully better — alert rules live in the database, contact points are first-class objects, and you can route different alert groups to different destinations without a separate Alertmanager process. For a Telegram notification when VRAM crosses 90% on the Ollama host, the setup is:

Create a Contact Point of type Telegram. You need a bot token (from @botfather) and your chat ID. Grafana will POST to the Telegram Bot API directly.
Create an Alert Rule on the VRAM panel: PromQL expression (nvidia_smi_memory_used_bytes / nvidia_smi_memory_total_bytes) * 100 > 90, evaluation interval 1m, pending period 2m (avoids false positives during model load spikes).
Assign that rule to a Notification Policy that routes to your Telegram contact point.

The webhook option works equally well if you want to fan out to multiple destinations or log alerts to n8n for further processing — set the contact point type to Webhook, point it at your n8n webhook trigger URL, and the full alert payload arrives as JSON you can route however you need.

OSS vs. Grafana Cloud

Grafana Cloud advertises a free tier and it is real — you get hosted Grafana, some Prometheus-compatible storage, and Loki ingestion up to defined limits. But for a self-contained single-workstation monitoring setup, the OSS version running locally has no meaningful missing features. Everything described above — community dashboards, custom panels, Unified Alerting, Telegram bots, webhook routing — ships in grafana/grafana-oss. Cloud starts making sense if you want metrics from machines that cannot expose a scrape endpoint to your Prometheus instance, or if you want Grafana-managed alerting SLA without running the container yourself. For a local homelab or a personal workstation running Ollama, the self-hosted OSS path is the simpler operational choice.

Tool 3: Netdata — When You Want Answers in 30 Seconds, Not 30 Minutes

Most monitoring stacks make you earn your dashboard. You provision Prometheus, write scrape configs, deploy Grafana, build panels, and somewhere around hour two you finally see a graph. Netdata skips all of that. Install it, hit port 19999, and you're already looking at per-second CPU steal, per-container network I/O, and NVMe latency — no YAML, no PromQL, no datasource configuration. That specific property is what makes it the right tool for an active incident rather than a weekly review.

The official one-liner gets you running in under two minutes on any Debian/Ubuntu or RHEL-family host:

# The --no-updates flag prevents automatic self-updates (good for production)
# --disable-telemetry is non-negotiable if this box has no outbound internet
wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh && \
  sh /tmp/netdata-kickstart.sh --no-updates --disable-telemetry --stable-channel

Skip --disable-telemetry and Netdata phones home with anonymous usage stats on startup. That's fine on an internet-connected dev box, but if you're installing on an air-gapped segment or a compliance-sensitive host, that outbound call will either block or violate policy. The Docker path is cleaner for reproducibility — mount /etc/netdata and /var/lib/netdata as volumes and you get a config-preserved, upgradeable container — but the native install gets eBPF working with less friction because the kernel headers are already present on the host.

The eBPF collector is the feature that actually separates Netdata from a Node Exporter + Grafana stack in a meaningful way. Node Exporter exposes aggregate block device and network interface stats. Netdata's eBPF module hooks at the kernel level and gives you per-process and per-cgroup breakdowns: which exact PID is issuing the most write() syscalls, which container is saturating your NVMe's write queue, which process is holding open the most file descriptors. On my 32GB workstation running several inference containers simultaneously, this is the fastest way to answer "why is disk latency spiking" without reaching for iotop or bpftrace manually. The collector is enabled by default on supported kernels (4.11+) — check /var/log/netdata/error.log if charts aren't appearing, because a missing linux-headers package is the most common silent failure.

The real constraint to plan around is retention. The OSS version keeps high-resolution data in a custom database (DBENGINE) and the default tier setup retains roughly a few days of per-second metrics before it starts downsampling or dropping. If you're doing capacity planning or want to correlate last Tuesday's memory creep with a deployment, that history won't be there. The two exits are: push to a Prometheus remote_write endpoint you already operate, or stream to InfluxDB. The config for remote_write is straightforward:

# /etc/netdata/exporting.conf
[prometheus_remote_write:my_prometheus]
    enabled = yes
    destination = prometheus-host:9090/api/v1/write
    # send only what you'll actually query — cardinality adds up fast
    send charts matching = system.* net.* disk.*

Without one of those backends, Netdata is a sharp diagnostic tool and a poor trend-analysis platform. That's not a criticism — it's the right mental model. Use it as the first thing you open during an incident and as the always-on overlay that tells you something is wrong before your alerting fires. Use Prometheus or VictoriaMetrics for the historical record. Running both on the same host costs you some RAM (Netdata sits around 150–300 MB depending on plugin count and DBENGINE tier config), but the operational split is clean: real-time clarity from Netdata, long-term context from your TSDB of choice.

Tool 4: Uptime Kuma — Endpoint Monitoring With Zero Complexity Tax

Most monitoring tools watch your server from the inside — they see CPU, memory, disk, process state. Uptime Kuma watches from the outside. It hits your endpoints the same way a user would, which means it catches a completely different failure class: your Nginx container is running, your app container is running, but the reverse proxy is returning 502s because the upstream socket path changed after a config reload. Prometheus won't fire an alert. Uptime Kuma will, within 60 seconds.

Getting it running takes one command. The named volume is non-negotiable — skip it and you lose all your monitor config on every container update:

docker run -d \
  --restart=always \
  -p 3001:3001 \
  -v uptime-kuma:/app/data \
  --name uptime-kuma \
  louislam/uptime-kuma:1

Hit http://your-server:3001, create an admin account, and you're configuring monitors. No YAML, no config files to maintain. The monitor types that matter for a typical self-hosting stack are specific: use HTTP(s) keyword match to hit your Ollama endpoint at http://localhost:11434/api/tags and assert the response body contains "models" — that verifies not just that the port is open but that the API is actually responding with valid data. Use TCP port for PostgreSQL on port 5432, since Postgres doesn't speak HTTP and you just need to know the socket is accepting connections. Use the Docker container monitor type to track container health status directly — it reads from the Docker socket, so a container that's running but in an unhealthy state (failed healthcheck) will register as down rather than up.

Notification routing is where Uptime Kuma earns its place in a real stack. Telegram bot notifications and Discord webhooks are both native, zero-plugin — you paste your bot token or webhook URL into a form and it works. The more useful option for a fully self-hosted setup is ntfy.sh, or better, a self-hosted ntfy instance. Uptime Kuma supports ntfy natively. You run ntfy in Docker, subscribe to a topic on your phone via the ntfy app, and now you have end-to-end self-hosted push alerts with no dependency on a third-party service. The config inside Uptime Kuma for a self-hosted ntfy instance is just your server URL, topic name, and optional auth token — takes 30 seconds.

The ceiling is real and worth naming clearly. Uptime Kuma stores no time-series data. There is no query language, no way to ask "what was my p95 response time over the last 30 days," no CPU or memory visibility. It keeps a response time graph per monitor but that's purely cosmetic — you can't alert on latency percentiles or correlate an endpoint slowdown with a memory spike. The correct mental model is: Uptime Kuma runs alongside Prometheus and your Grafana dashboards, not instead of them. Prometheus tells you what's happening inside your systems; Uptime Kuma tells you whether those systems are reachable from the network at all. They answer different questions. Running both on my workstation adds negligible overhead — Uptime Kuma idles under 100MB RAM — and the combination closes a blind spot that neither tool covers alone.

Tool 5: Zabbix — When the Complexity Is Actually Justified

Most monitoring tools make you choose between "easy to set up" and "actually scales." Zabbix lands in a third category: harder to set up than everything else on this list, but the architecture genuinely pays dividends once you cross the four-host threshold. A single Zabbix server can centrally manage agent configs, trigger thresholds, escalation policies, and dashboards for a NAS, a GPU workstation, a VPS, and a Raspberry Pi — from one place, with one schema. The operational model shifts from "log into each machine to check its monitoring" to "the monitoring comes to you."

The canonical self-hosted deployment is a Docker Compose stack: Zabbix server, the PHP frontend, and a PostgreSQL backend. The single most common first-boot failure is a credential mismatch between the database container and the Zabbix server container. Both must agree on the password, and they're set via separate env vars that are easy to set inconsistently:

services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_USER: zabbix
      POSTGRES_PASSWORD: your_secret_here   # set this
      POSTGRES_DB: zabbix

  zabbix-server:
    image: zabbix/zabbix-server-pgsql:alpine-7.0-latest
    environment:
      DB_SERVER_HOST: postgres
      POSTGRES_USER: zabbix
      ZBX_DBPASSWORD: your_secret_here      # must match exactly
      POSTGRES_DB: zabbix
    depends_on:
      - postgres

  zabbix-web:
    image: zabbix/zabbix-web-nginx-pgsql:alpine-7.0-latest
    environment:
      DB_SERVER_HOST: postgres
      POSTGRES_USER: zabbix
      ZBX_DBPASSWORD: your_secret_here      # same here
      POSTGRES_DB: zabbix
      PHP_TZ: America/New_York
    ports:
      - "8080:8080"

If the values drift, the server container exits immediately with a database connection error and the logs are sparse enough that the mismatch isn't obvious. Set these from a single .env file and reference them with variable substitution — don't type the password in three places manually.

The feature most people miss until they actually need it: Zabbix's network auto-discovery. You can define a rule that scans a CIDR range (say 192.168.1.0/24) on a schedule, checks for a running Zabbix agent on port 10050, and automatically registers matching hosts with a default template applied. This matters in practice when you're spinning up new Docker hosts or VMs — the new machine installs the agent, and within the next discovery interval it appears in your Zabbix dashboard without any manual "add host" workflow. For a homelab that sees regular churn, that compounds into real time saved. The auto-registration action is configured under Configuration → Discovery → Actions, and the default host template applied on registration is where most people spend time tuning.

The honest trade-off: the Zabbix web UI is genuinely dense. Terms like "hosts," "host groups," "templates," "items," "triggers," and "actions" have specific meanings that relate to each other in ways that aren't self-evident from the interface. Expect to spend real time with the docs before the mental model clicks. If your monitoring scope is a single workstation or two boxes, this overhead isn't justified. A stack of Prometheus with Node Exporter, Grafana for dashboards, and Uptime Kuma for endpoint checks covers the same ground — CPU, memory, disk, network, service availability — with a faster setup path and a UI that most people find navigable on first contact. Zabbix earns its complexity when the host count grows, when you want centralized agent config management, or when you need the built-in alerting escalation chains that Grafana only approximates through external integrations.

Picking the Right Stack for Your Setup

The most expensive mistake in self-hosted monitoring is over-engineering it before you understand your failure modes. A single GPU workstation running Ollama does not need the same stack as a six-host homelab with mixed workloads. The table below maps each tool against the dimensions that actually matter when you are choosing what to deploy — not marketing bullets, but the one thing that will make you regret the choice at 2am.

Tool

Deploy Complexity (1–5)

Metric Granularity

Alerting

Docker-Native

Host Resource Cost

Biggest Dealbreaker

Prometheus + Node Exporter

Very high

Via Alertmanager (separate config)

Yes

Low (scrape-based, idle is cheap)

No useful UI without Grafana; PromQL has a real learning curve

Grafana

2 (standalone) / 4 (full stack)

Depends on data source

Built-in (Grafana Alerting)

Yes

Low (visualization layer only)

Worthless without a metrics backend; becomes a maintenance surface on its own

Netdata

Very high (per-second, auto-discovered)

Built-in with ML anomaly detection

Yes

Medium (constant collection has CPU overhead)

Short local retention by default; cloud dependency for multi-node dashboards

Uptime Kuma

Low (availability + latency only)

Excellent (multi-channel, easy config)

Yes

Negligible

Not a metrics tool — tells you something is down, not why

Zabbix

High (agent + agentless, SNMP, JMX)

thorough (escalations, dependencies)

Partial (agent yes, server setup is heavy)

High (Postgres or MySQL backend required)

The UI and config model will eat a weekend before you see any value

Match your operator profile to a stack before you install anything. If you are running a single GPU workstation — Ollama, a few Docker services, maybe an n8n instance — Prometheus + Node Exporter + Uptime Kuma covers the surface area without overhead. Prometheus gives you the time-series record for CPU, memory, disk, and network; Uptime Kuma pings your endpoints and fires a notification when something stops responding. That is the entire job for a single-node setup. If your homelab spans multiple hosts with different roles, add Grafana for cross-host trending and start evaluating Zabbix only if you need host lifecycle management (auto-registration, templates across OS types, SNMP for network gear). Zabbix earns its complexity at that scale. Below it, the complexity is a tax with no refund.

If you have an incident right now and zero monitoring deployed, install Netdata first. One command, instant per-second visibility across CPU, memory, disk I/O, network, and running processes — no config file required for the base case. The migration path later is straightforward: keep Netdata for live incident debugging, layer Prometheus + Grafana on top for retention and trending, and use Uptime Kuma for availability checks. These three tools are not redundant — they occupy distinct roles. Uptime Kuma answers "is it up?", Prometheus + Grafana answers "what has it been doing over time?", and Netdata answers "what is happening right now and which process is the culprit?" Running all three on a single host is operationally reasonable and resource-light.

One gap none of these tools closes out of the box: VRAM utilization. If you are running Ollama, vLLM, or any inference workload, memory pressure on the GPU is your most likely OOM kill vector — and it will not show up in Node Exporter metrics without an additional exporter. The two realistic options are dcgm-exporter (NVIDIA's own, heavier, suited for multi-GPU) and nvidia_gpu_exporter (lighter, single-binary, easier to drop into a Compose file). Both expose Prometheus-compatible metrics. On my 32GB-VRAM workstation I run nvidia_gpu_exporter as a sidecar in the same Compose stack as Prometheus, then alert when VRAM utilization crosses a threshold before the OOM killer gets involved:

# docker-compose snippet — nvidia_gpu_exporter alongside Prometheus
  nvidia-exporter:
    image: utkuozdemir/nvidia_gpu_exporter:1.2.0
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    ports:
      - "9835:9835"
    restart: unless-stopped

# prometheus.yml scrape target
  - job_name: 'nvidia_gpu'
    static_configs:
      - targets: ['nvidia-exporter:9835']

# Alertmanager rule — fire before OOM, not after
  - alert: GPUMemoryPressure
    expr: nvidia_smi_memory_used_bytes / nvidia_smi_memory_total_bytes > 0.88
    for: 2m
    labels:
      severity: warning
    annotations:
      summary: "VRAM above 88% for 2m — check Ollama model load"

The 88% threshold is deliberate — it gives you a warning window before the kernel OOM fires at full saturation. Without this exporter in the stack, a Prometheus + Grafana setup gives you no signal that your inference workload is about to crash. That is the single monitoring gap most self-hosted LLM operators are running with today, and it is a straightforward fix once you know it exists.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

5 Self-Hosted Analytics Tools Worth Running on Your Own Hardware in 2026

우병수 — Fri, 31 Jul 2026 08:11:12 +0000

TL;DR: GA4's event model wasn't designed for people who want to understand their data — it was designed for people who want to feed Google's ad graph. The sampling kicks in the moment your traffic exceeds the free tier thresholds, which means the numbers you see in your dashboard are e

📖 Reading time: ~21 min

What's in this article

The Real Problem: You're Flying Blind or Paying Google to Watch You
How to Read This Comparison: Constraints That Force a Choice
Plausible Analytics: Minimum Footprint, Maximum Readability
Umami: When You Need Multi-Site Tracking Without the ClickHouse Tax
Matomo: Full-Featured but Operationally Expensive
PostHog: Product Analytics With a Real Event Pipeline
Grafana + a Data Source: When You Already Have the Metrics
When to Pick What: A Decision Matrix for Operators

The Real Problem: You're Flying Blind or Paying Google to Watch You

GA4's event model wasn't designed for people who want to understand their data — it was designed for people who want to feed Google's ad graph. The sampling kicks in the moment your traffic exceeds the free tier thresholds, which means the numbers you see in your dashboard are estimates dressed up as facts. Export to BigQuery is the official escape hatch, but it's throttled, requires a Google Cloud billing account, and you're still querying data that lives on infrastructure you don't control. For a small business operator, that's a fragile foundation: Google can deprecate the export format, change the schema, or restructure pricing, and your entire analytics workflow breaks overnight.

The self-hosted case isn't primarily a privacy argument, though that matters too. The real use is schema ownership. When you run your own analytics stack, you decide what an "event" means, how long raw data is retained, and whether you can run a GROUP BY at 2am without hitting a quota. You also control query latency — the difference between a ClickHouse instance on the same LAN as your dashboard and a round-trip to a managed SaaS API is measurable in seconds per page load when your queries get complex. That latency difference becomes load-bearing when you want to pipe analytics signals into automation workflows rather than just stare at charts.

The five tools covered here all have one thing in common: they run on commodity hardware without requiring a dedicated database cluster or a DevOps team. A single VPS with 2 vCPUs and 4GB RAM is enough to start with most of them. A home-lab Docker host is fine for lower-traffic sites. Each tool gets an honest resource cost — disk growth rate, RAM floor, CPU spikes during ingestion — and one dealbreaker that the README won't tell you upfront. No tool on this list is universally the right choice, and the honest answer is that the right pick depends on whether you need SQL access, funnel visualization, session replay, or lightweight script-tag deployment.

One angle that rarely gets covered in analytics tool comparisons: if you're already running automation pipelines, your analytics data shouldn't just sit in a dashboard — it should be a trigger source. A spike in 404 events, a drop in checkout completions below a threshold, a new referrer domain hitting your site — all of these are signals that can fire n8n workflows, send Slack alerts, or update a CRM record without a human checking a dashboard first. The infrastructure overlap between a self-hosted analytics backend and an event-driven automation stack is significant, and it's worth thinking about before you pick your schema. For the pipeline side of that equation, the Workflow Automation in 2026: n8n, Zapier, and Self-Hosted Pipelines guide covers how to wire those event sources into real workflows.

How to Read This Comparison: Constraints That Force a Choice

Most comparisons of analytics tools sort by feature count. That's the wrong axis. The real question is how much schema complexity and operational surface area you're willing to own. A page-view counter needs one table and a cron job. A full event pipeline needs a queue, a schema migration strategy, and someone to watch it at 2 AM when the ingestion worker silently backs up. Those are not the same class of problem, and picking the wrong tier will cost you more time than any missing feature ever would.

The hardware baseline throughout this comparison is a single Docker host: 2 vCPUs, 4 GB RAM, 20 GB SSD. That maps to a $6–12/month VPS (Hetzner CX22, DigitalOcean Basic, Vultr Regular) or a repurposed home-lab node running Debian or Ubuntu 22.04. If a tool routinely exceeds that envelope at idle — not under load, at idle — that will be flagged explicitly with numbers, not vague warnings. Some tools in this list are fine on that spec. One or two will push you toward 8 GB RAM before you've ingested a single real event.

The columns that actually drive the decision:

Minimum RAM at idle — what the process tree consumes after startup with no traffic. This is where most comparisons lie by omission.
Database backend — SQLite vs. Postgres vs. ClickHouse is not a detail, it's a maintenance contract. SQLite means zero ops and zero horizontal scale. ClickHouse means real performance on large datasets and real complexity on backup/restore.
Data export format — can you get your raw events out as CSV, JSON, or SQL dump without paying for a higher tier? Vendor lock-in in self-hosted tools usually lives here, not in the UI.
Self-hosted feature parity — some tools gate feature flags, funnels, or session replay behind their cloud plan even if you're running their Docker image. This is more common than the README implies.
Single biggest operational gotcha — the thing that doesn't appear until you've run it for a week: a migration that requires downtime, a default retention setting that fills your disk, a background job that pegs CPU on every page load.

The three tiers covered here map to real operational profiles. Lightweight page-view counters — Plausible and Umami — are single-binary or two-container deployments where the biggest decision is whether to use SQLite or Postgres. Full event pipelines — PostHog and Matomo — bring plugin ecosystems, session capture, and funnel analysis, but they also bring background workers, cache layers, and schema migrations you didn't ask for. Grafana sitting in front of a data source is a different category entirely: it doesn't collect events, it visualizes metrics you're already emitting, so its "analytics" story depends entirely on what's feeding it. Comparing those three tiers on the same feature grid produces misleading output. The sections below keep them in their lanes.

Plausible Analytics: Minimum Footprint, Maximum Readability

Plausible's Docker Compose stack is genuinely small — the official repo gives you a working docker-compose.yml on day one, and unlike most "self-hosted" analytics setups, you don't spend the first afternoon debugging missing env vars. The ClickHouse backend is pre-configured, the Elixir app boots cleanly, and you're ingesting hits within minutes. The catch that the README doesn't emphasize: that default ClickHouse config is a single-shard setup. Under roughly 10M monthly events it behaves fine. Above that, you're not dealing with a gradual slowdown — query times start spiking and you'll need to rethink the ClickHouse config before you hit the ceiling, not after.

Memory footprint at idle is honest for what it does. The Elixir app sits around 350 MB RSS, ClickHouse cold-starts near 600 MB. Both numbers are tolerable on a 2 GB VPS — until you get a traffic spike. ClickHouse will happily consume whatever RAM is available during a burst of aggregation queries. If you haven't dropped a max_memory_usage cap into your config.d/ override, expect it to balloon past 1.5 GB with no warning. The fix is straightforward but underdocumented:

<!-- /etc/clickhouse-server/config.d/memory.xml -->
<yandex>
  <max_memory_usage>1073741824</max_memory_usage>       <!-- 1 GB hard cap per query -->
  <max_memory_usage_for_all_queries>1610612736</max_memory_usage_for_all_queries>  <!-- 1.5 GB total -->
</yandex>

Mount that file into the ClickHouse container via your compose override and restart. Without it, on a small VPS, you're one dashboard refresh from an OOM kill.

The comparison with Plausible's paid cloud is frequently misrepresented. Funnels, revenue goals, and the Sites API are all present in CE — the real difference is operational: you own the ingestion pipeline end-to-end. One botched deploy, one misconfigured reverse proxy dropping the X-Forwarded-For header, and you're silently losing events with no alerting from Plausible itself. The cloud version has buffer layers and retries you never see. On CE, you build your own monitoring or you fly blind. A basic healthcheck hitting the /api/health endpoint on a 60-second cron is the minimum viable safety net:

# quick health probe — alerts if the Elixir app stops accepting requests
curl -sf https://your-plausible-domain.com/api/health || \
  notify-send "Plausible ingestion down"

The hardest limit for non-engineering users is raw event access. There is no "Export to CSV" button covering raw hits — the UI gives you aggregated views only. If you or anyone else needs row-level data, you go directly to ClickHouse via clickhouse-client or the HTTP interface. For an engineer that's fine; you can pull session-level data in seconds:

clickhouse-client --query \
  "SELECT session_id, pathname, country_code, timestamp
   FROM plausible_events_db.events
   WHERE domain = 'yoursite.com'
     AND toDate(timestamp) = today()
   LIMIT 500"

For a marketing hire or a business owner who just wants a spreadsheet, this becomes your support burden immediately. If anyone outside engineering needs to self-serve on raw data, factor in either building a thin query UI on top of ClickHouse or routing aggregated exports through something like Metabase pointed at the same database.

Umami: When You Need Multi-Site Tracking Without the ClickHouse Tax

Most self-hosted analytics tools that want to be taken seriously these days ship with ClickHouse as a hard dependency. That's a reasonable call for high-volume traffic, but it also means you're suddenly maintaining a columnar database engine alongside everything else, your backup story involves unfamiliar tooling, and your RAM budget just jumped by 2–4 GB minimum. Umami skips all of that. It writes directly to PostgreSQL or MySQL, which means if you're already running Postgres for anything else, Umami is just another schema in a database you already know how to operate. Backup is literally pg_dump. Restore is psql. There's nothing novel to learn.

The Docker Compose setup reflects that simplicity. Three services: the Next.js frontend/API app, a Postgres container, and nothing else. No Redis, no queue workers, no separate ingest service. Idle RAM on my box lands around 250 MB for the whole stack — that's lighter than a single Plausible container on a busy day. If you're running this on a $6 VPS or a shared homelab node, it won't crowd out your other workloads.

# minimal docker-compose.yml — production-ready starting point
services:
  umami:
    image: ghcr.io/umami-software/umami:postgresql-latest
    ports:
      - "3000:3000"
    environment:
      DATABASE_URL: postgresql://umami:${DB_PASSWORD}@db:5432/umami
      APP_SECRET: ${APP_SECRET}   # random string, not optional — sessions break without it
    depends_on:
      db:
        condition: service_healthy
    restart: unless-stopped

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: umami
      POSTGRES_USER: umami
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - umami_db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U umami"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  umami_db:

The v2 REST API is where Umami earns its place in an automation stack. Pulling stats into n8n or a cron job is a single authenticated GET call once you have a bearer token from POST /api/auth/login. From there, GET /api/websites/:id/stats?startAt=&endAt= returns pageviews, sessions, bounce rate, and visit duration as a flat JSON object — no pagination, no cursors, nothing exotic. In my n8n flows I use an HTTP Request node with the bearer token stored as a credential, call that endpoint on a schedule, and pipe the numbers directly into a Postgres insert or a Slack summary. The whole flow is under 10 nodes.

# grab a bearer token, then pull 7-day stats
TOKEN=$(curl -s -X POST https://your-umami-host/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"'$UMAMI_PASS'"}' \
  | jq -r '.token')

NOW=$(date +%s%3N)
WEEK_AGO=$(( NOW - 604800000 ))

curl -s "https://your-umami-host/api/websites/$SITE_ID/stats?startAt=$WEEK_AGO&endAt=$NOW" \
  -H "Authorization: Bearer $TOKEN" | jq .
# returns: {"pageviews":{"value":4821},"sessions":{"value":1203},...}

The performance ceiling is real and worth planning around before you commit. Once website_event crosses roughly 5 million rows, queries on the default schema start to drag — the dashboard date-range filters hit that table hard. The fix is a partial index on created_at scoped to your active website IDs, and ideally a monthly partitioning strategy if you're ingesting more than a few thousand events per day. Umami's migrations don't set this up for you, so it's manual work. And the dealbreaker is firm: there are no funnel views, no cohort analysis, no event sequencing. If your question is "how many users completed checkout within a session that started on the pricing page," Umami cannot answer that. It counts pageviews and custom events — cleanly, reliably, with low overhead — and that's the full scope of what it does.

Matomo: Full-Featured but Operationally Expensive

Matomo is the only self-hosted analytics tool that can walk into a GA4 feature comparison and not immediately lose. Goals, funnel reports, heatmaps, A/B testing, GDPR consent management — it's all there in the self-hosted version. That's a genuinely rare combination, and for a small business that migrated off Google Analytics and doesn't want to rebuild an analytics stack from scratch, Matomo is the obvious first stop.

The operational cost, though, is not small. Matomo runs on PHP-FPM + MySQL/MariaDB, and its database schema is wide. The two tables that will eventually cause you pain are log_visit and log_link_visit_action — they grow faster than you'd expect and don't self-prune without explicit configuration. Beyond storage, the bigger trap is the archiving cron. Matomo doesn't compute reports on the fly for historical data; it relies on a scheduled job to pre-aggregate everything:

# this must run reliably — add to crontab or a supervisor job
# if it times out silently, your reports will just show stale data with no error
*/5 * * * * /usr/bin/php /var/www/matomo/console core:archive --url=https://your-matomo-instance.com

The silent timeout is the failure mode that bites people. If your site has accumulated a large date range and the archive job hits PHP's max_execution_time or MySQL's wait_timeout before finishing, it exits without completing — and your dashboard just quietly shows yesterday's numbers forever. Fix this by setting max_execution_time = 0 in the PHP CLI config (not the web config), and tuning interactive_timeout and wait_timeout in MySQL to something above your expected archive duration. Also worth setting in config/config.ini.php:

[General]
# prevent archiving from triggering on browser requests — force cron-only
enable_browser_archiving_triggering = 0

# archive segments with data older than this many seconds
time_before_today_archive_considered_outdated = 900

RAM footprint at idle lands between 500 MB and 1 GB depending on your PHP-FPM pool size and MySQL buffer pool configuration. Once you cross roughly one million monthly pageviews, plan to dedicate 2 GB to this stack — and that's before you factor in the archiving job competing with live ingestion during peak hours. That's a non-trivial ask on a small VPS.

The dealbreaker for many operators is the plugin licensing model. Matomo's core is genuinely solid and open source, but the features most commonly cited as differentiators — heatmaps and session recording especially — require a paid license even on the self-hosted version. The pricing isn't obscene, but it fractures the "free GA4 replacement" argument immediately. If you need heatmaps on self-hosted Matomo, you're paying. That's a legitimate business model, but be clear-eyed about it before you migrate 30 properties onto this stack and then discover the plugin wall.

PostHog: Product Analytics With a Real Event Pipeline

PostHog occupies a different tier from Plausible or Umami — comparing them is like comparing a server access log analyzer to a full product intelligence platform. The self-hosted version ships with feature flags, session replay, heatmaps, funnel analysis, retention curves, path analysis, and an A/B testing engine, all wired into the same event pipeline. You're not stitching together four tools; it's one data model feeding every view. For a small product team that wants to run controlled rollouts and watch session replays without paying per-seat SaaS pricing, that consolidation is the entire argument.

The deployment story is where most people hit friction. PostHog is Kubernetes-first by architecture, and the official Helm chart makes that obvious. What saves smaller operators is the hobby deploy — a Docker Compose path that's actually maintained and documented, not an afterthought. The minimum spec is 4 GB RAM, but treat 8 GB as the real floor. ClickHouse is the hungry component here, same as in every other self-hosted analytics stack that prioritizes query speed. The hobby installer is a single curl-pipe-bash:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/PostHog/posthog/HEAD/bin/deploy-hobby)"

That script prompts for a domain, writes a .env, and brings up the compose stack. What it won't tell you upfront is that you're launching roughly 10 containers: ClickHouse, Kafka, Zookeeper, Redis, the Django app server, Celery beat, Celery worker, the plugin server (Node.js), an Nginx proxy, and a periodic worker. At idle, expect the stack to consume 3–4 GB RAM on the host — measured, not estimated. This is not a workload for a $6 VPS. A $24–$40/month dedicated instance with a real SSD is closer to the minimum viable host if you want headroom for actual event ingestion.

The self-hosted changelog is required reading before you commit. PostHog is honest in its docs about the feature gap: the cloud version ships AI-powered analysis, some newer product analytics views, and LLM observability tooling that the self-hosted version lags on, sometimes by months, sometimes indefinitely. The CHANGELOG.md in the repo and the self-hosted-specific release notes at posthog.com/docs/self-host/runbook will tell you exactly which features are missing. The gap isn't fatal for most small business use cases — funnel analysis, retention, and session replay are all present and maintained — but if you're evaluating PostHog specifically for its newer AI query features, test the cloud trial first and verify those features exist in the current self-hosted release before building a deployment around them.

Where PostHog makes sense over the lighter tools: you're building a product (SaaS, app, internal tool) and you need behavioral cohorts, not just page view counts. The JavaScript snippet and the posthog-js SDK capture custom events with properties, which means you can track button_clicked with { plan: "pro", screen: "checkout" } and then filter session replays to only users who hit that event before churning. That's the capability gap. If your analytics need is "how many people visited the pricing page," Umami costs you nothing and runs in 256 MB. If your need is "show me session replays of users who started checkout, enabled the coupon field, and didn't convert," PostHog is the only self-hosted option in this list that actually answers that question.

Grafana + a Data Source: When You Already Have the Metrics

Most analytics tool roundups slot Grafana in as if it competes with Plausible or Umami. It doesn't. Grafana visualizes data that already exists somewhere else — it has zero event ingestion capability of its own. No tracking pixel, no SDK, no JS snippet you embed on your site. If you don't already have a time-series store running, Grafana on its own is a dashboard with nothing to display. That's the first thing to establish, because it determines exactly who should use it.

The case for including it here is specific: if you're already running Plausible (which uses ClickHouse) or Umami (PostgreSQL), both databases can be wired directly into Grafana as data sources. What that unlocks is correlation dashboards that no SaaS analytics product touches — overlay your Plausible pageview counts against Prometheus node CPU metrics, then drop in Loki annotations for deployment events. A traffic spike that coincides with a deploys and a CPU peg tells you something a standalone analytics UI never would. On my own setup I pull Umami's Postgres data alongside node_exporter metrics in the same Grafana instance, and seeing both on one timeline is genuinely useful for distinguishing "the site got traffic" from "the site got slow."

The resource story is split in two. Grafana itself is light — around 150 MB RSS at idle, negligible CPU unless you're rendering a lot of panels simultaneously. The real cost is the stack behind it:

Grafana + Prometheus + Loki together run roughly 1.5–2 GB before you add any exporters. That's meaningful on a small VPS.
Grafana + existing Plausible ClickHouse adds almost nothing — you're querying a store you already pay for.
Alerting via Grafana Alertmanager adds another moving part; if you only want dashboards, disable it and save the complexity.

The Docker Compose addition to an existing Plausible stack is minimal. You expose the ClickHouse port internally and point Grafana at it:

# add to your existing plausible docker-compose.yml
  grafana:
    image: grafana/grafana-oss:10.4.2
    ports:
      - "3001:3000"
    volumes:
      - grafana_data:/var/lib/grafana
    environment:
      # disable public signup — non-negotiable on a public host
      GF_AUTH_DISABLE_LOGIN_FORM: "false"
      GF_USERS_ALLOW_SIGN_UP: "false"
    depends_on:
      - plausible_db  # ensures ClickHouse is up first

volumes:
  grafana_data:

Then in the Grafana UI, add a ClickHouse data source (requires the grafana-clickhouse-datasource plugin) pointed at plausible_events_db on the internal Docker network. You can query Plausible's events table directly with raw SQL panels — no intermediate ETL needed. The dealbreaker remains firm though: if you're starting from scratch with no existing metrics infrastructure and just need to know how many people visited your site, skip Grafana entirely and start with Umami or Plausible. Grafana earns its spot only when you're layering business observability on top of operational monitoring that's already running.

When to Pick What: A Decision Matrix for Operators

The mistake most operators make is picking analytics software based on feature lists rather than operational fit. The question isn't which tool has the most capabilities — it's which one you'll still be running cleanly six months from now without having touched the config. These five tools solve genuinely different problems, and using the wrong one costs you either maintenance overhead or missing data.

Pick Plausible CE if you want a production-ready, low-maintenance page analytics stack on a single VPS and you're comfortable dropping into ClickHouse when you need raw data. Plausible's Docker Compose setup is stable, the resource footprint is small, and the default dashboard covers 90% of what a content operator actually needs day-to-day. The trade-off: the API surface is narrow, and when you want something it doesn't expose in the UI, you're writing ClickHouse SQL directly — which is powerful but not a casual afternoon task.
Pick Umami if you need multi-site tracking under a single install, your infrastructure already runs PostgreSQL, and you want a clean REST API for pulling stats into automation pipelines. Of every tool in this list, Umami is the easiest to integrate with n8n or a TypeScript/Node publishing engine running on PM2 — the /api/websites/:id/stats endpoint returns structured JSON with no ceremony. Token auth works cleanly, response times are fast on even modest hardware, and the schema is simple enough to query directly if the API doesn't expose what you need.
Pick Matomo if you have a real compliance requirement — GDPR audit trail, configurable consent management, or a data processing agreement you need to document — or if you have non-technical stakeholders who need GA-equivalent reports without learning a new interface. Matomo is the heaviest of the five operationally, but it's the only one where a marketing person can sit down and feel at home immediately. The plugin ecosystem also means you can add heatmaps, A/B testing, and form analytics without bolting on separate tools.
Pick PostHog if you're operating a product rather than a content site and you need funnel analysis, feature flags, and session replay in one place. This is not a casual self-host — budget at least 8 GB RAM for the host, expect ClickHouse to be the memory floor, and read the resource requirements before committing. The payoff is that PostHog gives you product analytics depth that none of the others touch: cohort retention, event-level autocapture, and feature flag targeting that actually closes the loop between deploy and behavior data.
Layer Grafana on top of any of the above if you're already running a home-lab or server monitoring stack and want business metrics in the same dashboard as node exporter, Caddy access logs, or container health. Grafana doesn't replace any of these tools — it reads from their underlying databases (Postgres, ClickHouse, MySQL) via datasource plugins and lets you build unified panels. On my workstation setup, having a single Grafana board showing site traffic alongside GPU utilization and n8n workflow success rates is genuinely more useful than toggling between four separate UIs.

One cross-cutting signal worth flagging: if your stack is already PostgreSQL-heavy, Umami and Matomo both run on it natively, which means one fewer moving part in your backup and restore procedures. If you're already committed to ClickHouse for something else — or willing to learn it — Plausible and PostHog both use it as their event store, and the query power you get in return is significant. Don't introduce a second database engine just because a tool looks appealing; the operational cost compounds quietly until a disk fills or a migration breaks.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

Nagios + Grafana on Self-Hosted Hardware: A Real Monitoring Stack Without the Cloud Tax

우병수 — Wed, 29 Jul 2026 08:11:29 +0000

TL;DR: The failure mode that actually bites self-hosted operators isn't a dramatic crash with error messages everywhere — it's a silent service exit at 2am that nobody notices until the automation pipeline produces zero output at 9am. A container stops responding, a cron job silently s

📖 Reading time: ~24 min

What's in this article

The Problem: Flying Blind on Your Own Hardware
Architecture Decision: Why Both Tools Instead of One
Deploying Nagios Core in Docker
Deploying Prometheus and Grafana in Docker
Service Checks That Actually Matter for a Self-Hosted AI Stack
Notification Routing and Avoiding Alert Fatigue
Gotchas That Cost Real Time

The Problem: Flying Blind on Your Own Hardware

The failure mode that actually bites self-hosted operators isn't a dramatic crash with error messages everywhere — it's a silent service exit at 2am that nobody notices until the automation pipeline produces zero output at 9am. A container stops responding, a cron job silently skips, a disk fills to 100% and writes start failing quietly. No alert fires because there's no alerting. No timeline exists because there's no metrics history. You're left doing forensics on stale logs trying to reconstruct what happened and when. That's the specific problem this guide solves.

Cloud monitoring tools don't fit this situation for two compounding reasons. First, cost: Datadog, New Relic, and similar platforms price per host or per metric volume. A homelab or small self-hosted stack with eight to twelve machines gets expensive fast, and the pricing tiers are designed for teams with budgets, not solo operators paying out of pocket. Second, and more important for the threat model: these tools work by shipping your internal metrics — service names, CPU patterns, network topology, error strings — to someone else's infrastructure. If you're running services on your own hardware specifically to avoid that kind of data exposure, sending detailed telemetry about your internal systems to a SaaS vendor undermines the whole premise.

A minimal but real monitoring stack needs three layers that most incomplete setups are missing at least one of. Host-level checks cover the basics that will kill you if ignored: CPU sustained above threshold, RAM pressure causing swap, disk partitions approaching full, load average spiking. Service-level checks go further — they verify that the thing you care about is actually responding, not just that the process is listed in ps. A Docker container can show as Up 3 days while the application inside it has deadlocked and stopped accepting connections. Visual dashboards handle the third failure mode: slow degradation. Disk growing at 2GB/day, RAM usage climbing 5% weekly — these don't trigger threshold alerts until it's already an emergency, but they're obvious on a time-series graph if you look.

The stack covered here is Nagios Core for check execution and alerting, Prometheus for metrics collection, and Grafana for dashboards — all running in Docker on the same LAN segment as the workloads they monitor. That last point matters: the monitoring stack has no dependency on external connectivity. If your internet goes down, you still get alerts on the local network. Nagios handles the "is this service up right now?" question with active checks and notification routing. Prometheus handles the "what has this looked like over the past 30 days?" question with time-series storage. Grafana sits in front of Prometheus and makes that history readable without writing PromQL from scratch every time. These tools have been around long enough that their rough edges are documented, their failure modes are known, and their Docker images are stable.

Architecture Decision: Why Both Tools Instead of One

The tempting shortcut is picking one tool and bending it to cover both jobs. Grafana has alerting. Prometheus has threshold rules. Nagios has a built-in web dashboard. Every one of those "just use this one thing" paths leads to a system that half-works in two directions instead of working well in one. The clean split is this: Nagios owns the pager, Grafana owns the screen. Once you commit to that boundary, the entire stack stops fighting itself.

Nagios Core does threshold-based alerting with a notification pipeline that's been production-hardened for decades. You define a check, set warning and critical thresholds, wire up a contact group, and when a disk hits 90% you get paged — via email, a PagerDuty webhook, or a Slack incoming hook. That pipeline is synchronous, stateful, and has built-in escalation logic. What Nagios cannot do is show you what CPU utilization looked like at 3am last Tuesday, or whether your container memory has been creeping up 200MB/week for the last month. It has no time-series store. Asking it to draw trends is like asking a smoke detector to show you a temperature graph.

That gap is exactly what the Prometheus + Grafana layer fills. On my 32GB workstation running several Ollama model-serving processes, the GPU utilization dashboard in Grafana is the only thing that's caught a slow VRAM leak before it hard-crashed the inference process. The symptom wasn't a threshold breach — VRAM never hit 100% in a single scrape — it was a 48-hour upward slope that only showed up when you looked at the sparkline. Nagios would have slept through that entirely. Grafana + nvidia_smi_exporter scraped every 30 seconds made it visible in under a day.

The overlap trap catches most people at the Grafana alerting step. Grafana can absolutely fire alerts. Prometheus has ALERT rules. If you turn both on alongside Nagios, you end up with three systems that can all page you for the same event — and worse, you'll find yourself tuning suppression rules across three different UIs to stop the duplication. The maintenance cost compounds fast. The rule is: one pager, one dashboard. Silence Grafana's alerting entirely. Do not configure Alertmanager to send pages. Let Nagios own all notification routing, period. This sounds like a constraint but it's actually a relief.

On resource cost: Nagios Core is almost free at runtime. Expect under 100MB RSS even with a few hundred active checks. Prometheus is a different story. With a 15-day retention window, scraping 5 hosts every 15 seconds at moderate metric cardinality (node_exporter default is around 800 metrics per host), you'll accumulate roughly 2–4GB of TSDB data per day depending on how many containers and custom exporters you add. Before you deploy, size your --storage.tsdb.retention.time and make sure the target volume can absorb it. On a home lab with a spinning disk, that's manageable. On a VPS with expensive block storage, it becomes a cost decision. The flag to control it in your compose file looks like this:

command:
  - '--config.file=/etc/prometheus/prometheus.yml'
  - '--storage.tsdb.path=/prometheus'
  - '--storage.tsdb.retention.time=15d'   # tune before first deploy
  - '--storage.tsdb.retention.size=20GB'  # hard cap as a safety net

Set both flags. The time-based retention is the primary control; the size cap is the circuit breaker if your cardinality explodes after adding a new exporter. Without the size cap, a misconfigured exporter emitting high-cardinality labels (per-request URLs are the classic offender) can fill a disk inside 48 hours on an otherwise healthy setup.

Deploying Nagios Core in Docker

The jasonrivers/nagios image is the community-maintained option that's actually kept up — the official Nagios dockerfiles have gone stale. Pin to a specific tag. 4.4.14 is stable as of this writing. The reason to pin isn't pedantry: Nagios config syntax has genuinely shifted between minor versions, particularly around the use inheritance directive and host template parsing. Running latest means a surprise rebuild can silently break your config validation, and you won't know until you restart the container and nagios refuses to start.

Here's a minimal docker-compose.yml that mounts your config and state directories separately — this matters because you want to edit config files outside the container lifecycle, and you don't want a container rebuild wiping your status history:

services:
  nagios:
    image: jasonrivers/nagios:4.4.14
    container_name: nagios
    restart: unless-stopped
    ports:
      # Bind only to your LAN interface — never 0.0.0.0 on a monitoring host
      - "192.168.1.10:8080:80"
    volumes:
      - ./nagios/etc:/opt/nagios/etc      # all config lives here
      - ./nagios/var:/opt/nagios/var      # status.dat, logs, retention
    environment:
      - NAGIOSADMIN_USER=nagiosadmin
      - NAGIOSADMIN_PASS=changeme_before_first_run

The port binding 192.168.1.10:8080:80 is doing real work. If you bind to 0.0.0.0, the Nagios web UI is reachable from anywhere the host is reachable, which on a typical home lab includes WAN if you've got port forwarding open for other services. Bind to your actual LAN IP and that attack surface disappears. Your first run goal is simple: get the UI green on the default localhost host before touching anything else.

Once the container is up, add your first real host. Drop a file at ./nagios/etc/objects/hosts.cfg — the container's default nagios.cfg includes cfg_dir=/opt/nagios/etc/objects so any .cfg file there gets picked up:

define host {
    host_name               fileserver
    alias                   Main File Server
    address                 192.168.1.20
    check_command           check-host-alive
    max_check_attempts      3
    check_period            24x7
    notification_period     24x7
    contacts                nagiosadmin
}

Validate before restarting — this catches syntax errors without a full restart cycle:

docker exec nagios /opt/nagios/bin/nagios -v /opt/nagios/etc/nagios.cfg

If validation passes, restart the container and confirm the host shows green in the web UI. Don't pile on service checks until you've confirmed ICMP reach works. A host that shows as DOWN before you've added any services tells you immediately whether it's a network or config problem.

For real host metrics — disk, CPU, running processes — you need NRPE on each monitored machine. The nagios-nrpe-server package on Ubuntu/Debian installs the daemon and a default config at /etc/nagios/nrpe.cfg. The critical setting that most setups get wrong initially is allowed_hosts. It must contain the IP your Nagios container uses when making outbound connections — which is the Docker host's LAN IP if you're using bridge networking, not the container's internal IP:

# /etc/nagios/nrpe.cfg on the monitored host
allowed_hosts=127.0.0.1,192.168.1.10   # 192.168.1.10 = your Nagios Docker host

# These ship by default — verify they're present and paths are correct for your distro
command[check_users]=/usr/lib/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_disk]=/usr/lib/nagios/plugins/check_disk -w 20% -c 10% -p /
command[check_zombie_procs]=/usr/lib/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 150 -c 200

On the Nagios container side, you need a check_nrpe command definition. The jasonrivers/nagios image ships with the NRPE plugin already installed, so you only need the command object in your config:

# ./nagios/etc/objects/commands.cfg — append this block
define command {
    command_name    check_nrpe
    # -H is the target host, -c is the command name defined in nrpe.cfg
    command_line    /opt/nagios/libexec/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}

Then wire it into a service definition for your host:

define service {
    host_name               fileserver
    service_description     Disk /
    check_command           check_nrpe!check_disk
    check_interval          5
    max_check_attempts      3
    check_period            24x7
    notification_period     24x7
    contacts                nagiosadmin
}

A common failure mode here: NRPE returns "Connection refused" even with correct allowed_hosts. Check whether nagios-nrpe-server is actually listening — ss -tlnp | grep 5666 on the monitored host. Some Ubuntu installs require systemctl enable --now nagios-nrpe-server explicitly. The other gotcha is firewall rules: if ufw is active, you need ufw allow from 192.168.1.10 to any port 5666 on the monitored host before NRPE will respond.

Deploying Prometheus and Grafana in Docker

The Scrape Chain Before You Touch Compose

The mental model that makes this whole stack click: Prometheus pulls, it never receives pushes. Every host you want to monitor needs something listening on a port that Prometheus can scrape. For bare-metal or VM system metrics, that's node_exporter on port 9100. For per-container metrics on your Docker host, that's cadvisor on port 8080 — except if Nagios is already bound to 8080 (a common default for its web UI), you'll hit a silent conflict. Run cadvisor on 8081 instead and adjust your prometheus.yml accordingly. The conflict won't crash anything; Docker will just refuse to start cadvisor, and you'll have a gap in your metrics you won't notice until you wonder why all your container panels are empty.

Node exporter runs best as a systemd service on bare-metal hosts rather than a Docker container, because getting accurate host-level metrics from inside a container requires --pid=host --net=host --privileged — at which point you've basically given it host access anyway. The systemd approach is cleaner:

# Download the current release (check github.com/prometheus/node_exporter/releases for latest)
wget https://github.com/prometheus/node_exporter/releases/download/v1.8.2/node_exporter-1.8.2.linux-amd64.tar.gz
tar xvf node_exporter-1.8.2.linux-amd64.tar.gz
sudo cp node_exporter-1.8.2.linux-amd64/node_exporter /usr/local/bin/

# /etc/systemd/system/node_exporter.service
[Unit]
Description=Prometheus Node Exporter
After=network.target

[Service]
User=node_exporter
ExecStart=/usr/local/bin/node_exporter
Restart=on-failure

[Install]
WantedBy=multi-user.target

Cadvisor on the Docker host is a different story — it needs access to the Docker socket and cgroup filesystem, so running it as a container with the right mounts is the standard approach. Pin the version; the latest tag on cadvisor has historically had breaking changes between minor versions:

cadvisor:
  image: gcr.io/cadvisor/cadvisor:v0.49.1
  container_name: cadvisor
  privileged: true
  ports:
    - "8081:8080"   # left side changed to avoid Nagios conflict
  volumes:
    - /:/rootfs:ro
    - /var/run:/var/run:ro
    - /sys:/sys:ro
    - /var/lib/docker/:/var/lib/docker:ro
    - /dev/disk/:/dev/disk:ro
  restart: unless-stopped

prometheus.yml and the Retention Flag You Shouldn't Skip

The default retention in Prometheus is 15 days, which sounds fine until you realize the default storage path (/prometheus inside the container) will eat through disk silently. On a host with a small root partition, this shows up as a full disk with no obvious culprit. Always pass --storage.tsdb.retention.time=15d explicitly in your Docker command or compose file — not because it changes the default, but because it documents your intent and makes the behavior predictable when you later tune it. A minimal but complete prometheus.yml:

# prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets:
          - 'host1.lan:9100'
          - 'host2.lan:9100'
          # add every host running node_exporter here

  - job_name: 'cadvisor'
    static_configs:
      - targets:
          - 'localhost:8081'   # matches the remapped port above

And the Prometheus container block in compose, with the retention flag explicit and storage on a named volume so data survives container rebuilds:

prometheus:
  image: prom/prometheus:v2.53.0
  container_name: prometheus
  command:
    - '--config.file=/etc/prometheus/prometheus.yml'
    - '--storage.tsdb.path=/prometheus'
    - '--storage.tsdb.retention.time=15d'   # explicit, not assumed
    - '--web.enable-lifecycle'               # lets you reload config via POST /-/reload
  volumes:
    - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
    - prometheus_data:/prometheus
  ports:
    - "9090:9090"
  restart: unless-stopped

Grafana: Provisioning Over Manual Config

If you configure the Prometheus data source manually through the Grafana UI, the next time you rebuild the container it's gone. Provisioning via YAML solves this — Grafana reads datasource definitions from a mounted directory at startup. The setup requires two pieces: the compose block and a provisioning file on disk.

grafana:
  image: grafana/grafana:11.1.0
  container_name: grafana
  environment:
    - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD}  # set in .env, never hardcode
    - GF_USERS_ALLOW_SIGN_UP=false
  volumes:
    - grafana_data:/var/lib/grafana
    - ./grafana/provisioning:/etc/grafana/provisioning:ro
  ports:
    - "3000:3000"
  depends_on:
    - prometheus
  restart: unless-stopped

# grafana/provisioning/datasources/prometheus.yml
apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090   # container name resolves via Docker DNS
    isDefault: true
    editable: false               # prevents accidental UI changes from persisting

The editable: false flag is worth setting if you want the provisioned config to be the source of truth. Without it, someone can modify the datasource in the UI, and the next restart will silently overwrite their changes with the provisioned values — which is confusing to debug. Make the behavior explicit either way.

Dashboards 1860 and 14282: Useful Starting Points, Not Blind Trusts

Dashboard ID 1860 (Node Exporter Full) and 14282 (Docker and system monitoring via cadvisor) are real, published dashboards on grafana.com. Both are worth importing as a starting point — they save hours of panel configuration. Import them via Grafana's UI at Dashboards → Import → Enter ID, or provision them as JSON files. The honest caveat: both dashboards include panels that query metrics your specific exporter version or configuration may not expose. A panel showing "No data" isn't broken Grafana — it means the metric the panel queries (node_filesystem_avail_bytes with a specific label, for instance) doesn't exist in your scrape. Audit the panel queries before you put these dashboards on a wall display or alert on them. Delete panels that consistently return no data rather than leaving a dashboard full of empty graphs, which trains you to ignore gaps.

1860 common gap: Some network interface panels use label matchers that don't match non-standard interface names (like enp3s0 vs eth0). Edit the panel query's device label filter to match your actual interface names.
14282 common gap: Container name labels in cadvisor changed format across versions. If you're on cadvisor v0.47+, the name label on container metrics dropped the leading slash — old dashboard queries using =~"/.*" may not match.
Both dashboards: Check which Grafana version they target. A dashboard built for Grafana 9 using the old Prometheus query editor may render differently in Grafana 11, particularly around panel transformations.

Service Checks That Actually Matter for a Self-Hosted AI Stack

Most Nagios setups default to checking whether a port is open and calling it done. For a self-hosted AI stack, that misses every failure mode that actually bites you — the Ollama process that's listening but has unloaded all models due to memory pressure, the n8n container that restarts every 90 seconds but always passes a "is it running?" check, the model storage volume that hits 95% because you pulled a 70B weight without thinking.

Checking Ollama Properly: /api/tags, Not Just Port 11434

A TCP port check against 11434 will return OK even if Ollama is mid-crash or stuck in an unresponsive state after an OOM kill. The /api/tags endpoint is the right target — it forces a real HTTP round-trip through the Ollama HTTP layer and returns a 200 with a JSON body listing loaded models. If that returns a 200, the server is actually alive. Use check_http with the -u and -e flags:

# In your Nagios commands.cfg or services config
define command {
    command_name  check_ollama_api
    command_line  $USER1$/check_http -H $HOSTADDRESS$ -p 11434 \
                  -u /api/tags -e "200 OK" \
                  -w 5 -c 10
    # -w/-c are response time thresholds in seconds
    # Ollama can be slow to respond if a model is mid-load
}

define service {
    use                   generic-service
    host_name             ai-workstation
    service_description   Ollama API Health
    check_command         check_ollama_api
}

Set your warning threshold at 5 seconds and critical at 10. A healthy Ollama responding to /api/tags with no models loading takes under 200ms. If it's crossing 5s, something is wrong with the model server even if the port is technically open. On my 32GB box, slow responses to that endpoint have reliably preceded an OOM event by a few minutes — the server was thrashing trying to keep a model resident.

Disk Checks Against the Model Storage Mount Specifically

Ollama model weights live in ~/.ollama/models by default, and a single model file can run anywhere from 4GB (a quantized 7B) to 70GB+ for a full-precision 70B. The root partition check that ships with most Nagios templates is useless here — model storage is almost always on a separate mount or at minimum a different logical volume. Check the actual mount point where model files land:

# Using check_disk against a dedicated model storage mount
define command {
    command_name  check_model_storage
    command_line  $USER1$/check_disk -w 20% -c 10% \
                  -p /mnt/models
    # -w 20% means warn when 20% FREE remains (i.e., 80% used)
    # Nagios check_disk uses free space, not used — easy to get backwards
}

define service {
    use                   generic-service
    host_name             ai-workstation
    service_description   Model Storage Disk
    check_command         check_model_storage
}

Note the inverted logic: check_disk warns when free space drops below the threshold, so -w 20% triggers the warning when you're at 80% used, and -c 10% fires critical at 90% used. If you write it as -w 80% thinking "warn at 80%", you'll fire warnings constantly because 80% free space almost never exists on a volume full of model weights. Get this backwards once and you'll spend 20 minutes debugging why your disk check is always critical.

Docker Container State via NRPE: Catching Silent Exits

Docker's built-in healthcheck and Nagios's check_procs both miss the same failure: a container that exited cleanly with code 0 and had no restart policy. The Docker daemon reports the engine as healthy, the process table shows no zombie, and everything looks fine until you notice n8n hasn't processed a workflow in 3 hours. A custom NRPE script catches this:

#!/bin/bash
# /usr/local/lib/nagios/plugins/check_docker_container
# Run via NRPE on the host — requires docker group membership for the nagios user

CONTAINER=$1

if [ -z "$CONTAINER" ]; then
    echo "UNKNOWN: No container name specified"
    exit 3
fi

RUNNING=$(docker inspect --format='{{.State.Running}}' "$CONTAINER" 2>/dev/null)
EXIT_CODE=$(docker inspect --format='{{.State.ExitCode}}' "$CONTAINER" 2>/dev/null)

if [ $? -ne 0 ]; then
    echo "CRITICAL: Container $CONTAINER not found"
    exit 2
fi

if [ "$RUNNING" = "true" ]; then
    echo "OK: $CONTAINER is running"
    exit 0
else
    echo "CRITICAL: $CONTAINER is not running (exit code: $EXIT_CODE)"
    exit 2
fi

# In nrpe.cfg on the monitored host
command[check_n8n_container]=/usr/local/lib/nagios/plugins/check_docker_container n8n
command[check_ollama_container]=/usr/local/lib/nagios/plugins/check_docker_container ollama

Add the nagios user to the docker group on the host, or this script will fail silently with a permission error that $? returns as non-zero but the output is empty. That produces CRITICAL: Container n8n not found even when the container is fine — a confusing false positive that wastes time.

n8n Crash-Loop Detection: Uptime Window Check

The /healthz endpoint on n8n (exposed on port 5678 by default) returns a 200 if the process is responsive. That's necessary but not sufficient. A container that crash-loops every 90 seconds with a --restart=always policy will pass the healthcheck right after each restart — it's technically healthy for 60 of those 90 seconds. A separate check against container uptime catches this:

#!/bin/bash
# /usr/local/lib/nagios/plugins/check_container_uptime
# Warns if a container has been running for less than N seconds — 
# designed to catch restart loops, not replace the running-state check

CONTAINER=$1
MIN_UPTIME=${2:-300}  # default: warn if running less than 5 minutes

STARTED=$(docker inspect --format='{{.State.StartedAt}}' "$CONTAINER" 2>/dev/null)

if [ $? -ne 0 ]; then
    echo "UNKNOWN: Cannot inspect $CONTAINER"
    exit 3
fi

# Convert ISO8601 timestamp to epoch — requires GNU date
STARTED_EPOCH=$(date -d "$STARTED" +%s 2>/dev/null)
NOW_EPOCH=$(date +%s)
UPTIME=$((NOW_EPOCH - STARTED_EPOCH))

if [ "$UPTIME" -lt "$MIN_UPTIME" ]; then
    echo "WARNING: $CONTAINER only up for ${UPTIME}s (threshold: ${MIN_UPTIME}s) — possible restart loop"
    exit 1
fi

echo "OK: $CONTAINER up for ${UPTIME}s"
exit 0

# NRPE command entry
command[check_n8n_uptime]=/usr/local/lib/nagios/plugins/check_container_uptime n8n 300

Pair this with a 2-minute Nagios check interval and a 1-occurrence alert threshold. A single restart isn't always alarming — Docker will restart a container after an update or a host reboot. The signal you want is the check firing on consecutive runs: if n8n's uptime resets between each 2-minute polling cycle, something is actively wrong. Set max_check_attempts 3 and notification_interval 0 in the service definition so you get one alert per event, not a flood every 2 minutes during a crash loop.

Notification Routing and Avoiding Alert Fatigue

The most common failure mode in self-hosted monitoring isn't missing an alert — it's generating so many that you start unconsciously filtering them yourself. Once your brain learns to skim past Nagios emails, the whole system is broken. The fix isn't smarter alerts; it's routing the right severity to the right channel before you ever hit that point.

Set up two contacts: one for email via Postfix or an SMTP relay container, one for Slack via a shell script wrapping curl. The Slack contact should receive CRITICAL only. Email handles WARNING. This isn't just aesthetics — a Slack ping at 2am demands attention, an email at 6am gets triaged with coffee. Here's the shell script pattern for the Slack contact:

#!/bin/bash
# /usr/local/bin/notify-slack.sh
# Called by Nagios with env vars set by the contact definition
curl -s -X POST "$SLACK_WEBHOOK_URL" \
  -H 'Content-type: application/json' \
  --data "{
    \"text\": \"*[$NAGIOS_NOTIFICATIONTYPE]* $NAGIOS_HOSTNAME: $NAGIOS_SERVICESTATE — $NAGIOS_SERVICEOUTPUT\"
  }"

Wire it into the contact definition like this:

define contact {
    contact_name                    ops-slack
    service_notification_commands   notify-service-by-slack
    service_notification_options    c          ; CRITICAL only
    host_notification_options       d,u        ; DOWN and UNREACHABLE only
    service_notification_period     24x7
}

define command {
    command_name    notify-service-by-slack
    command_line    /usr/local/bin/notify-slack.sh
}

The notification_interval trap catches nearly every solo operator eventually. Nagios defaults to re-notifying every 30 minutes on a still-failing check. If you're running overnight model jobs on a GPU box and something goes sideways at 11pm, you will wake up to a dozen Slack pings by morning and start associating the channel with noise. For a single-operator setup, push that to 60 or 120 minutes on non-critical services:

define service {
    host_name                       gpu-workstation
    service_description             VRAM Usage
    check_command                   check_vram_util
    notification_interval           120   ; re-notify every 2 hours, not 30 min
    first_notification_delay        5
    notification_options            w,c,r
    contacts                        ops-email
}

Scheduled downtime is the other lever people skip until they regret it. Patching a box at midnight without setting downtime first means Nagios fires host-down and service-down notifications for every check on that host — easily 30-40 emails depending on how many services you're monitoring. The Nagios web UI handles this under System → Schedule Downtime, but for anything you're automating — cron-triggered reboots, maintenance scripts — hit the CGI endpoint directly:

# Schedule 90 minutes of downtime for host "gpu-workstation" starting now
curl -s -u nagiosadmin:yourpassword \
  "http://localhost/nagios/cgi-bin/cmd.cgi" \
  --data "cmd_typ=55&cmd_mod=2&host=gpu-workstation&com_author=cron&com_data=planned+reboot&trigger=0&start_time=$(date +'%m-%d-%Y+%H:%M:%S')&end_time=$(date -d '+90 minutes' +'%m-%d-%Y+%H:%M:%S')&fixed=1&hours=1&minutes=30&childoptions=0"

One thing worth keeping straight: silence via downtime is not the same as filtering via notification options. Downtime suppresses alerts while still running checks and recording state. A contact with service_notification_options w stripped out simply never routes WARNING notifications to that contact — the check still fires, the state still changes, Grafana still graphs it. You want both mechanisms. Downtime for planned maintenance, filtered routing for permanent severity separation. Conflating them leads to gaps where you think you're monitoring something but no channel is actually wired to receive it. For operators building broader pipelines where this notification layer feeds into webhook triggers or self-healing automation, the architecture overlaps with general workflow tooling — the Workflow Automation in 2026: n8n, Zapier, and Self-Hosted Pipelines guide covers where those systems connect.

Gotchas That Cost Real Time

The most expensive one: Nagios silently stops checking things when you reload a bad config, and the container keeps running like nothing happened. Before you ever run docker exec nagios kill -HUP 1, verify the config file first:

# Run this inside the container or via exec — never skip it
docker exec nagios /usr/local/nagios/bin/nagios -v /opt/nagios/etc/nagios.cfg

# Expected tail of good output:
# Total Warnings: 0
# Total Errors:   0
# Things look okay - No serious problems were detected during the pre-flight check

A typo in a host definition, a missing use template reference, or a circular dependency in service groups will cause the reload to fail. Nagios won't crash — it keeps running the last known good config. But if you made that config change because a host was added, that host gets zero checks, zero alerts, and you won't notice until something on it actually breaks and nobody pages you. Make nagios -v a pre-commit hook or at minimum a manual step before every reload.

Prometheus cardinality is the slower-burning problem. cAdvisor emits labels like container_label_com_docker_compose_service, image, interface, and several others per time series. On a stable set of long-running containers that's manageable. The moment you start running ephemeral containers — n8n's Execute Command nodes, one-shot ETL jobs, anything that spins up and tears down per workflow run — you accumulate unique label value combinations that never get cleaned up within the retention window. Query performance on container_cpu_usage_seconds_total starts to drag. The fix is metric_relabel_configs in your Prometheus scrape config, applied before the data lands in TSDB:

scrape_configs:
  - job_name: cadvisor
    static_configs:
      - targets: ['cadvisor:8080']
    metric_relabel_configs:
      # Drop per-interface network metrics you're not dashboarding
      - source_labels: [__name__]
        regex: 'container_network_(receive|transmit)_(packets|errors|dropped)_total'
        action: drop
      # Strip high-cardinality compose labels from all metrics
      - regex: 'container_label_com_docker_(compose_config_hash|compose_version|swarm.*)'
        action: labeldrop

Drop rules are evaluated before storage, so they reduce active series immediately. Don't wait until Prometheus is already slow to add these — by then you're also dealing with a bloated WAL.

Grafana's provisioning behavior bites almost everyone once. If you mount a dashboard JSON into /etc/grafana/provisioning/dashboards/ and Grafana picks it up on startup, that dashboard is now managed by the provisioning system. Edit it in the UI, save it, restart the container — your edits are gone. Grafana overwrites the dashboard from the file on startup. The options are: keep the JSON file as the single source of truth and make all edits there (export from UI, update the file, restart), or after the initial import remove the provisioning config entry so Grafana owns it going forward. What you cannot do is treat both as writeable simultaneously. Pick a lane.

NRPE firewall rules through Docker are genuinely tricky and the standard advice fails here. Running ufw allow from 192.168.1.50 to any port 5666 on the monitored host feels correct, but Docker's iptables manipulation means the Nagios container's traffic arrives on the host with the bridge network's IP, and your ufw rule may never match it. The correct test is always from inside the Nagios container, not from the host itself:

# Test from inside the Nagios container
docker exec -it nagios nc -zv 192.168.1.200 5666

# If that fails but the host-level nc works, you have a Docker FORWARD chain issue
# Check what Docker inserted:
iptables -L DOCKER -n --line-numbers
iptables -L FORWARD -n --line-numbers | grep 5666

If you're using ufw, the reliable path is adding the NRPE allow rule directly via iptables targeting the Docker bridge subnet, or switching the monitored host's NRPE binding to a specific interface and exposing the port in the compose file with explicit host binding. Testing from the Docker network namespace rather than from the bare host catches this class of bug before you spend an hour wondering why Nagios reports all NRPE checks as unreachable.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

5 Lightweight Self-Hosted Error Trackers That Actually Replace Sentry on Your Own Hardware

우병수 — Mon, 27 Jul 2026 08:09:56 +0000

TL;DR: The free tier cap isn't an inconvenience — it's a trap. A mid-traffic side project running into a bad deploy can exhaust Sentry's hosted event quota within the first hour of the incident.

📖 Reading time: ~16 min

What's in this article

Why Sentry's Hosted Version Becomes a Problem at Scale
The 5 Tools: Quick Comparison Before the Detail
GlitchTip: The Closest Drop-In for Sentry SDK Users
Errbit: For Teams Running Ruby or Wanting Airbrake Compatibility
Signoz: When You Need Traces and Metrics Alongside Errors
Highlight.io: Session Replay Plus Errors in One Self-Hosted Stack
Picking the Right Tool for Your Setup

Why Sentry's Hosted Version Becomes a Problem at Scale

The free tier cap isn't an inconvenience — it's a trap. A mid-traffic side project running into a bad deploy can exhaust Sentry's hosted event quota within the first hour of the incident. That's the exact window where you need error visibility most, and instead you're watching the dashboard show zeros while your users are hitting 500s. Upgrading mid-incident isn't a workflow. Paying for headroom you don't normally need just to survive the occasional spike is a recurring tax that compounds across multiple projects.

Self-hosting the official Sentry distribution solves the quota problem but trades it for infrastructure complexity that's wildly out of proportion to the task. The docker-compose manifest ships with over 20 services — Kafka, ClickHouse, Redis, Celery workers, Snuba, the Relay ingestion pipeline, and more. The install script refuses to proceed with less than 3 GB of RAM allocated, and that's before you've ingested a single event. Running this on a shared VPS or home-lab machine that already hosts other workloads means you're either constantly fighting OOM kills or dedicating a node exclusively to error tracking. That's a hard justification to make when the application generating the errors is itself a small project.

The core mismatch is what developers actually want from a self-hosted error tracker versus what Sentry's architecture was built to deliver. Source maps, readable stack traces, release tagging, environment filters, basic alerting — that's the real list. Kafka exists in the official stack because Sentry processes billions of events per month across their SaaS offering. That throughput requirement drives architectural decisions that then get inherited by anyone self-hosting, regardless of whether they're handling a hundred events a day or a million. You end up running a small data platform to power what should be a single-purpose tool.

For this comparison, "lightweight" means something specific and testable:

Deployable as a single Docker container or a Compose stack with no more than three or four services
Under 1 GB RAM at idle — low enough to share a 2 GB VPS with a running application
Postgres or SQLite as the only stateful dependency — no Kafka, no ClickHouse, no separate queue infrastructure
Functional source map support and structured stack traces — not just raw JSON event dumps

That constraint set cuts out a lot of candidates. Several "Sentry alternatives" in blog posts still pull in Redis plus a task queue plus a separate ingestion service the moment you enable any real feature. The ones worth running on a single box are the ones that made deliberate architectural decisions to stay small — and those decisions show up immediately in the Compose file line count and the docker stats output after a day of running.

The 5 Tools: Quick Comparison Before the Detail

The most useful thing to know upfront: these five tools split cleanly into two categories that serve different operators. Two of them — GlitchTip and Errbit — exist specifically to be Sentry-compatible backends. Point your existing DSN at them and you're done. The other three — SigNoz, Highlight.io, and Baselime (now AWS-native) — are broader observability stacks where error tracking is one tab among several. Picking the wrong category means either ripping out instrumentation you don't need to touch, or paying for a full observability platform when you just want a crash log.

Sentry SDK compatibility is the deciding filter for most self-hosters. If your app already ships @sentry/node, sentry-python, or the browser @sentry/browser package, a DSN-compatible backend costs you zero instrumentation rework — you change one environment variable. If you pick a non-compatible platform, you're re-instrumenting every service, which is a real cost on a multi-service setup. Don't let a feature list override that operational reality.

Here's the comparison across the dimensions that actually matter when you're planning a self-hosted deployment:

Tool          Min RAM    Backing Store         Sentry SDK Compat   Source Maps   Biggest Dealbreaker
-----------   --------   -------------------   -----------------   -----------   -------------------------------------------
GlitchTip     512 MB     Postgres + Redis       Yes (drop-in DSN)   Yes           Grouping logic lags behind Sentry's
Errbit        256 MB     MongoDB                Yes (drop-in DSN)   No native     No timeline view; Ruby-era UX
SigNoz        4 GB+      ClickHouse             No                  Yes           Heavy stack; ClickHouse eats disk fast
Highlight.io  2 GB+      Postgres + ClickHouse  No                  Yes           Requires full session replay pipeline
Baselime       —         AWS (not self-hosted)  No                  Yes           Not actually self-hostable; AWS-only

A few numbers worth expanding on: SigNoz's ClickHouse dependency is the operational weight you're accepting. ClickHouse is a columnar store optimized for high-throughput append workloads — great for traces and metrics at scale, but it will consume disk at a rate that surprises operators coming from Postgres-only stacks. Expect to set aggressive TTL policies on your spans table within the first week or you'll watch a modest event volume balloon into tens of gigabytes. GlitchTip's 512 MB floor is real and reproducible on a $6/month VPS, making it the only option here that doesn't demand a dedicated box. Errbit's 256 MB claim holds only if you keep MongoDB lean — in practice, with a few weeks of error history, you're looking at 1 GB+ for the Mongo data directory alone.

Source map support is the other sharp dividing line, especially for frontend-heavy teams. GlitchTip handles uploaded source maps through its release artifact API, which is compatible with the standard sentry-cli upload workflow. Errbit has no built-in source map processing — stack traces from minified JS land in your inbox exactly as mangled as they came in, which makes it a poor fit for any project shipping a bundled frontend. If you're also evaluating AI-assisted debugging tooling alongside these, see our guide on AI Coding Tools in 2026: Cloud Copilots vs Local Models for the broader context on where automated analysis fits into an error triage workflow.

GlitchTip: The Closest Drop-In for Sentry SDK Users

The migration story is almost suspiciously simple: swap one URL. GlitchTip uses the exact same DSN format as Sentry, which means the difference between pointing at sentry.io and pointing at your own box is literally a hostname change. SENTRY_DSN=https://key@sentry.io/123 becomes SENTRY_DSN=https://key@your-glitchtip.host/123 — no SDK version bump, no new client library, no configuration schema to learn. If you're already running Sentry SDKs across a handful of services, GlitchTip is the only self-hosted option where you aren't also rewriting instrumentation.

The full stack is a single Django app, Postgres, and Redis. That's it. A working docker-compose.yml stays under 50 lines with room to spare, and after the containers warm up, idle RAM hovers in the 200–300 MB range total — not per service. The minimum viable setup needs three environment variables before docker compose up -d will get you anywhere useful:

# .env — minimum required before first boot
SECRET_KEY=replace-with-a-long-random-string
DATABASE_URL=postgres://glitchtip:pass@db:5432/glitchtip
GLITCHTIP_DOMAIN=https://errors.yourdomain.com

The non-obvious one to set immediately — before you ingest a single event — is GLITCHTIP_MAX_EVENT_LIFE_DAYS. GlitchTip has no automatic event expiry enabled out of the box. On a busy application this means your Postgres volume grows without any ceiling, and on a cheap VPS with 20–40 GB of disk you'll eventually hit a full-disk condition that takes the whole container stack down silently. The fix is a one-liner in your env file, but the docs don't surface it prominently:

# add to .env — without this, Postgres grows unbounded
GLITCHTIP_MAX_EVENT_LIFE_DAYS=90

Performance monitoring exists — GlitchTip does accept transaction data from Sentry SDKs — but treat it as a checkbox, not a feature. You get basic transaction lists and error-rate summaries. What you don't get: session replay, profiling, span-level flame graphs, or anything from the Sentry performance tab that you'd actually open during an incident. If your team uses Sentry primarily to catch and group unhandled exceptions, GlitchTip covers that workload cleanly. If someone on your team has Sentry's profiler tab bookmarked, they'll notice the gap within a day.

Errbit: For Teams Running Ruby or Wanting Airbrake Compatibility

The first thing to understand about Errbit is that it does not speak Sentry's protocol. It implements the Airbrake v2 API — which means you're reaching for the airbrake gem or airbrake-js, not any Sentry SDK. That's a hard architectural constraint, not something you toggle in a config file. If your codebase already has Sentry SDK calls scattered through it, Errbit is not a drop-in swap. But if you're on a legacy Rails monolith that's been using the Airbrake notifier for years, Errbit is essentially a self-hosted backend that already matches your client setup exactly.

The Docker compose setup is straightforward: the official errbit/errbit image paired with a mongo:6 container. No Redis dependency, which keeps the stack simpler than most alternatives. The real resource cost is MongoDB — expect 300–500 MB resident memory on a quiet instance just from the Mongo process sitting there. That's not a dealbreaker, but it's the number to check against your VPS tier before committing.

services:
  errbit:
    image: errbit/errbit:latest
    environment:
      - RACK_ENV=production
      - MONGO_URL=mongodb://mongo:27017/errbit
      - SECRET_KEY_BASE=changeme_generate_with_openssl_rand
      - EMAIL_FROM=errors@yourdomain.com
    ports:
      - "8080:8080"
    depends_on:
      - mongo

  mongo:
    image: mongo:6
    volumes:
      - errbit_mongo:/data/db
    # no auth config here — add --auth and a keyfile for production

volumes:
  errbit_mongo:

Error grouping uses backtrace fingerprinting, which handles Ruby and Rails stack traces well — the frames are stable, human-readable, and consistent between deploys as long as your gem versions don't churn. Where this breaks down is minified JavaScript. Without source maps pre-processed server-side before errors arrive, Errbit groups JS errors against mangled frame addresses that change every build. You get a new "unique" error on every deploy rather than a deduplicated stream. If JavaScript error tracking is a significant part of your needs, that friction is real and not easily patched around.

The honest best-fit profile for Errbit: a Rails monolith already wired to the Airbrake notifier, an infrastructure where MongoDB is already running for something else (so the memory cost is already paid), and a team that doesn't need Slack-level integrations or sophisticated alerting rules. It's minimal, it's stable, and it doesn't require you to retrain anyone who's been using Airbrake. Outside that context — especially if you're running a polyglot stack or need strong JavaScript support — the API lock-in will cost you more than the self-hosting saves.

Signoz: When You Need Traces and Metrics Alongside Errors

The biggest conceptual shift with SigNoz isn't the UI or the query language — it's that errors don't arrive via a Sentry DSN at all. SigNoz is OpenTelemetry-native, which means exceptions surface as OTel log and exception events flowing through the collector pipeline. If you've been using @sentry/node or any Sentry SDK, swapping to SigNoz means ripping that out and replacing it with the OTel SDK stack. That's not a config change; it's a real migration. Expect to touch every service that currently initializes Sentry, replace the SDK initialization, and wire up the exporter endpoint.

# Replacing Sentry init with OTel in a Node.js service
# Before (Sentry):
# Sentry.init({ dsn: "https://...", tracesSampleRate: 1.0 });

# After (OTel):
npm install @opentelemetry/sdk-node \
  @opentelemetry/auto-instrumentations-node \
  @opentelemetry/exporter-otlp-grpc

// instrumentation.ts — runs before anything else via --require
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-grpc';

const sdk = new NodeSDK({
  // Point at your SigNoz OTel collector, not a Sentry DSN
  traceExporter: new OTLPTraceExporter({
    url: 'grpc://your-signoz-host:4317',
  }),
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start();
// Unhandled exceptions now appear as span events in SigNoz,
// not as Sentry issues — the grouping model is completely different

The Docker Compose install is where "lightweight" gets complicated. The official stack pulls in ClickHouse, a query service, the OTel collector, an alertmanager, and several supporting containers — the total is typically 8+ containers. SigNoz's own docs recommend a 4 GB RAM floor just for the stack itself. On a shared 4 GB VPS that's already running Nginx, Postgres, and your app, SigNoz will win the memory contest and everything else will lose. ClickHouse under indexing load is particularly aggressive about buffer allocation. This isn't a dealbreaker — it just means SigNoz belongs on a dedicated node or a home-lab machine where you have 8+ GB free to hand it.

# Checking what the Compose stack actually pulls
git clone https://github.com/SigNoz/signoz.git
cd signoz/deploy/docker/clickhouse-setup
docker compose config --services
# Expect: clickhouse, query-service, frontend, alertmanager,
#         otel-collector, otel-collector-metrics, logspout, ...
# Count them before you commit to a VPS tier

What you actually gain over simpler alternatives like GlitchTip is real correlated observability: a single failing request shows you the exception, the full distributed trace, the service map hop where latency spiked, and infrastructure metrics on the same timeline. That correlation is the feature. When an error fires at 2am, you're not manually cross-referencing three dashboards — the trace is attached to the exception event. The query interface also lets you slice arbitrary log fields rather than just exception type and message, which matters once you're past basic error grouping and want to understand patterns across deployments or regions.

The operational cost for that capability is real. When the stack fails to start cleanly, you're not reading one log — you're reading ClickHouse startup output and the OTel collector logs simultaneously to figure out which dependency failed to become healthy first. A common failure mode is ClickHouse not being ready when the query service attempts its initial schema migration; the fix is usually just waiting and restarting the query service container, but figuring that out the first time takes longer than it should. Use SigNoz when you have spare hardware capacity and you've outgrown pure error tracking — when you need traces to explain why errors happen, not just that they happened.

Highlight.io: Session Replay Plus Errors in One Self-Hosted Stack

Session replay baked into the same error pipeline is the actual differentiator here — not in a marketing sense, but in a debugging sense. When a user hits a TypeError: Cannot read properties of undefined in your React app, you don't just get a stack trace; you get an rrweb recording of exactly what they clicked, scrolled, and typed before the crash. Correlating errors to sessions without a separate tool like LogRocket or FullStory is the whole reason to look at Highlight.io seriously.

The self-hosted stack is honest about its complexity. The Docker Compose setup pulls in Postgres (session metadata, user data), ClickHouse (event storage and aggregation), and either MinIO or an S3-compatible store for the raw replay chunks. That's not bloat for the sake of it — ClickHouse is genuinely the right engine for time-series event queries at replay scale, and object storage is the only sane place to put binary replay data. But the practical consequence is that you need 6+ GB RAM as a floor before your actual application runs. ClickHouse alone wants 2-4 GB to behave, MinIO needs headroom, and the Highlight app containers add more on top. If you're thinking about slotting this onto a $6/mo VPS, the math doesn't work. This belongs on a home-lab box or a cloud instance with at least 8 GB dedicated to the observability stack.

# The core of the self-hosted compose — what you're actually committing to:
services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: highlight
  clickhouse:
    image: clickhouse/clickhouse-server:23.12
    # expects ~2GB RAM minimum under real query load
    ulimits:
      nofile:
        soft: 262144
        hard: 262144
  minio:
    image: minio/minio
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: highlight
      MINIO_ROOT_PASSWORD: changethis
  highlight:
    image: ghcr.io/highlight/highlight:latest
    depends_on: [postgres, clickhouse, minio]

The SDK situation is where migration cost becomes real. Highlight ships @highlight-run/node for the backend and highlight.run for the browser — neither is a drop-in for the Sentry SDK. There's no Sentry-compatible DSN endpoint, no captureException alias that just works. You're doing a proper SDK swap: remove @sentry/node, install @highlight-run/node, update your error boundary wrappers, update your environment config, and then learn a new dashboard. For a mature app with Sentry wired into a dozen places, that's a half-day minimum, not a config swap. The upside is the product you get is genuinely richer — network request timelines, console logs, and the replay are all first-class in the same view.

The honest use-case boundary: if your errors are predominantly backend — Node.js workers, API services, queue consumers — Highlight's resource overhead is hard to justify. GlitchTip or Signoz will give you error aggregation for a fraction of the RAM. Highlight earns its place specifically when you're debugging frontend-triggered failures where the stack trace alone doesn't tell you enough. A user reports a blank screen on checkout; the replay shows them resizing the browser window at a specific breakpoint right before the React hydration error fires. That's the scenario where six gigabytes of supporting infrastructure actually pays for itself.

Picking the Right Tool for Your Setup

The honest answer most "comparison" posts dodge: for a solo operator who just wants Sentry-compatible error tracking without babysitting infrastructure, GlitchTip on a 1 GB VPS with Postgres is the default. Drop in your existing Sentry SDK, set one environment variable, put Nginx in front, and you're done. The config that actually matters:

# docker-compose env block — the one variable most people miss
GLITCHTIP_MAX_EVENT_LIFE_DAYS=90   # without this, the DB grows unbounded
SECRET_KEY=your-random-64-char-string
DATABASE_URL=postgres://glitchtip:pass@db:5432/glitchtip
EMAIL_URL=smtp://user:pass@smtp.yourhost.com:587

Nginx in front handles TLS termination, and Certbot handles renewal. The entire ops surface is one Compose file and a cron for docker compose exec web python manage.py migrate after upgrades. A 1 GB VPS handles low-to-moderate error volumes without complaint — Postgres is the only datastore, so backups are a single pg_dump command. That simplicity is the point.

Two legitimate reasons to deviate from that default. If you're running a legacy Ruby app already wired to Airbrake, pick Errbit and accept MongoDB. Rewriting SDK calls across a mature Rails codebase to hit a different endpoint costs more than running Mongo ever will. MongoDB on a single node with a scheduled mongodump is not a complex ops burden — the friction of fighting SDK compatibility is. Conversely, if you've got a spare home-lab node with 8+ GB RAM and need traces plus metrics alongside errors, SigNoz is the correct answer. Instrument everything with OpenTelemetry from day one:

# Node.js OTel bootstrap — same collector config works for SigNoz today,
# any OTLP-compatible backend tomorrow
OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4317
OTEL_SERVICE_NAME=my-api
OTEL_TRACES_EXPORTER=otlp
OTEL_METRICS_EXPORTER=otlp

The OTel abstraction means you're not locked to SigNoz — if you outgrow it or migrate, the instrumentation code doesn't change, only the collector endpoint does. That matters more than any feature SigNoz ships next quarter.

The one case where none of the above applies: frontend apps where session replay is a hard requirement. Highlight.io is the only self-hosted option that ships replay, error tracking, and logging as a single deployable stack. Building replay yourself on top of GlitchTip or Errbit means maintaining a separate rrweb pipeline, a storage layer for recordings, and a playback UI — that's a significant side project. Highlight.io requires a dedicated host with meaningful RAM headroom, but if replay is genuinely non-negotiable for debugging user-reported issues, the ops cost is justified and there's no real alternative in the self-hosted space.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

Grafana + Prometheus on a Raspberry Pi: Build a Telemetry Dashboard That Actually Stays Up

우병수 — Fri, 24 Jul 2026 08:10:51 +0000

TL;DR: The failure mode nobody warns you about with Raspberry Pi clusters: the service dies, you SSH in twenty minutes later, and dmesg shows the CPU was thermally throttling for two hours before the crash. No alert fired.

📖 Reading time: ~21 min

What's in this article

The Problem: Flying Blind on Low-Power Hardware
Architecture Before You Install Anything
Installation: Docker Compose Stack with Real Config Files
node_exporter: What to Enable and What to Skip
Building the Dashboard: Panels That Catch Real Problems
Alerting Without AlertManager Complexity
Gotchas That Will Cost You an Afternoon

The Problem: Flying Blind on Low-Power Hardware

The failure mode nobody warns you about with Raspberry Pi clusters: the service dies, you SSH in twenty minutes later, and dmesg shows the CPU was thermally throttling for two hours before the crash. No alert fired. Nothing logged the temperature climb. You were flying completely blind while your node was quietly strangling itself.

Thermal throttling on ARM chips is insidious because it doesn't fail loudly — it just makes everything slower in a way that looks like bad code. The Pi 4 and Pi 5 will both throttle aggressively when the SoC hits 80°C, dropping clock speed without any visible error. SD card I/O saturation is the same story: iowait climbs past 40%, your service's write queue backs up, and the symptom looks like a memory leak or a hung process. Memory exhaustion on a 2GB board will OOM-kill a process while leaving the system technically "up," which means your monitoring check passes but your actual workload is dead. Without instrumentation, you're diagnosing all of this from crash artifacts instead of catching the curve before it peaks.

The minimal signal set you actually need to answer these questions is smaller than most guides suggest. Four metrics cover the majority of failure modes on constrained hardware:

CPU temperature — the raw SoC temp in Celsius, not CPU utilization, which can look fine while the chip throttles
Memory pressure — available memory plus swap usage, because free memory is misleading when the kernel is aggressively reclaiming cache
Disk I/O wait — iowait percentage and queue depth, which catches SD card saturation before it becomes a hang
Network throughput — bytes in/out per interface, which surfaces runaway log shippers or unexpected traffic on headless nodes

Prometheus plus Grafana wins on this hardware class for a specific structural reason: the scrape model puts the collection burden on the exporter, not a persistent agent. node_exporter on ARM64 idles around 10–20MB RSS during normal operation — it wakes up, serves a scrape, and goes back to sleep. Compare that to the Telegraf + InfluxDB stack, where Telegraf runs continuous collection intervals and InfluxDB's write-ahead log and compaction processes compete for memory on the same 1–4GB budget. The Prometheus TSDB is also local-queryable without a separate query engine, which matters when you're running Grafana on the same Pi rather than a separate host. The tradeoff you accept is that Prometheus's local retention gets expensive on disk past 15 days of high-cardinality data — on an SD card, that's a real constraint you'll need to tune. If you're also running AI tooling or heavier inference workloads on the same Pi or a companion machine, the AI Coding Tools in 2026: Cloud Copilots vs Local Models guide covers the broader toolchain context worth reading before you allocate RAM budgets across competing services.

One gotcha that isn't in the official docs: Prometheus's default scrape interval of 15 seconds is too coarse to catch transient I/O spikes on slow SD cards. A spike that saturates the card for 8 seconds and then clears will be missed entirely between two 15-second samples. Drop the scrape interval to 5 seconds for disk metrics specifically — you can do this per-job in the scrape config without affecting everything else — and your storage costs increase proportionally, so plan retention accordingly before you commit.

Architecture Before You Install Anything

The topology decision matters more than any config tweak you'll make later. Running Prometheus and Grafana directly on the Pi 4 is the obvious path, but on a 4 GB model you're looking at Prometheus sitting around 80–150 MB RSS at rest, Grafana adding another 100–150 MB, plus whatever else the OS is doing. That's 300–400 MB committed before you've scraped a single metric. It's workable, but if the Pi is already doing something else — running a media server, a home automation stack, anything that actually uses memory — you'll feel it. The cleaner split is running Prometheus and Grafana on a separate host (a spare x86 machine, another Pi, even a cheap VPS) and dropping only node_exporter on the Pi itself. node_exporter idles under 10 MB RSS and barely touches the CPU. That's the architecture I'd recommend if you have the option — the Pi becomes a pure data source, and your monitoring stack doesn't compete with whatever workload the Pi is actually supposed to run.

Scrape interval is where people make a quiet mistake with SD cards. The Prometheus default of 15s is sane for most metrics, but if you want to catch thermal throttling events — the Pi starts throttling at 80°C and the kernel can flip the throttle flag and clear it within seconds — you need 5s or you'll miss the event entirely in the time series. The cost of dropping to 5s isn't compute, it's writes. At 15s scrape with node_exporter exposing roughly 700–900 metrics, Prometheus writes compressed blocks to disk in chunks, not per-scrape, so the write amplification is lower than you'd expect. But at 5s you're tripling the in-memory accumulation rate and flushing chunks more frequently. On a USB SSD this is a non-issue — the write endurance is effectively unlimited at this workload. On an SD card running ext4 with default mount options, sustained small writes shorten card life measurably over months. If you're stuck with SD, at minimum mount noatime and point Prometheus's TSDB data directory at a USB stick or external drive.

The retention math is worth doing before you provision storage. Prometheus with node_exporter at a 15s scrape interval generates somewhere in the 200–400 MB range per month of compressed TSDB data — the variance comes from how many network interfaces, disks, and CPUs you're exposing. The default retention is 15 days, which keeps you under 200 MB in most node_exporter-only setups. If you add custom metrics or drop the scrape interval, recalculate. Set --storage.tsdb.retention.time explicitly regardless — if you don't set it, you're relying on the compiled default surviving across version upgrades, which it won't always do cleanly. A reasonable explicit flag for a Pi with a USB SSD:

--storage.tsdb.path=/data/prometheus \
--storage.tsdb.retention.time=30d \
--storage.tsdb.retention.size=2GB

The retention.size flag is your safety net. Prometheus will evict oldest blocks before it hits that limit, so even if your scrape rate spikes, you won't fill the drive silently.

Docker Compose versus bare-metal is an honest trade-off, not a clear winner. Compose gives you version-pinned images, a single docker-compose.yml that you can commit to git, and teardown/rebuild in under two minutes — that reproducibility pays off the first time you corrupt a config and need to roll back. The cost is real: the Docker daemon itself plus the overhead of two containers adds roughly 80–120 MB RSS that you don't pay on bare metal. On a 4 GB Pi running nothing else, that's tolerable. On a 2 GB Pi or a Pi that's doing actual work, that overhead pushes you toward bare-metal installs via the official APT repos or prebuilt ARM binaries from the Prometheus GitHub releases page. If you go bare-metal, pin your versions explicitly in whatever provisioning script you use — prometheus-2.51.2.linux-armv7 not "latest" — because the ARM binary naming conventions have shifted between releases and an unattended upgrade can silently pull a mismatched build.

Installation: Docker Compose Stack with Real Config Files

Pinning image versions on ARM64 is non-negotiable. The latest tag for several Prometheus ecosystem images has a documented lag on ARM64 — the amd64 manifest updates first, and if your Pi pulls during that window, you either get a stale layer or a failed pull. Beyond that, unpinned images mean a docker compose pull three months from now silently changes behavior. Pick a version, write it down, upgrade deliberately.

Here's the full docker-compose.yml. Every image is pinned to a specific minor version, volumes are named (not anonymous), and the network is explicit so Prometheus can reach node-exporter by service name:

version: "3.8"

networks:
  telemetry:
    driver: bridge

volumes:
  prometheus_data:
  grafana_data:

services:
  node-exporter:
    image: prom/node-exporter:v1.8.0
    container_name: node_exporter
    restart: unless-stopped
    networks:
      - telemetry
    pid: host                          # needed to expose real host-level CPU/mem metrics
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.sysfs=/host/sys'
      - '--path.rootfs=/rootfs'
      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'

  prometheus:
    image: prom/prometheus:v2.51.2
    container_name: prometheus
    restart: unless-stopped
    networks:
      - telemetry
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention.time=15d'  # explicit retention, default is 15d but state it
    user: "65534:65534"                # nobody:nogroup — see ownership note below
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana:10.4.2
    container_name: grafana
    restart: unless-stopped
    networks:
      - telemetry
    volumes:
      - grafana_data:/var/lib/grafana
      - ./provisioning:/etc/grafana/provisioning:ro
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=changeme
      - GF_USERS_ALLOW_SIGN_UP=false
    ports:
      - "3000:3000"

The prometheus.yml scrape config below uses a job_label on the target so that if you later add a second Pi, you're not staring at unlabeled localhost:9100 entries in every query. Label the host now, even if it's a single node:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "node"
    static_configs:
      - targets: ["node-exporter:9100"]   # service name resolves on the telemetry network
        labels:
          host: "pi-living-room"          # change per node; shows up in every metric series

Grafana provisioning is the piece most tutorials skip, and it's what keeps your Prometheus datasource alive through docker compose down && docker compose up. Without it, the datasource lives only in the SQLite database inside the named volume — which is fine until you recreate the volume or restore from backup without also restoring the DB. Create ./provisioning/datasources/prometheus.yaml:

apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090   # container-to-container on the shared network
    isDefault: true
    editable: false               # prevent accidental UI edits from diverging from file
    jsonData:
      timeInterval: "15s"        # match global scrape_interval or Grafana will warn

Two ownership problems will bite you on first start if you use bind mounts instead of named volumes — and even with named volumes on some Raspberry Pi OS setups. Prometheus runs as UID 65534 (nobody) and will refuse to start if it can't write to /prometheus. If you ever switch to a bind mount for the data directory, run sudo chown -R 65534:65534 ./prometheus_data before docker compose up. Grafana uses UID 472, which is non-obvious and not the same as any default user on Raspberry Pi OS. Same fix: sudo chown -R 472:472 ./grafana_data if you're bind-mounting. Named volumes sidestep this because Docker manages the directory and applies container ownership automatically — but the moment you ls -la /var/lib/docker/volumes/ and try to manually drop files in, you're back to the same problem.

node_exporter: What to Enable and What to Skip

The default node_exporter install ships with roughly 50 collectors active. On a Pi, maybe a dozen of those are useful. The rest are either irrelevant (NFS, RAID), cause unnecessary disk writes on SD cards, or pull kernel subsystems that don't exist on ARM. Getting the collector list right upfront saves you from scraping noise and from quietly shortening the life of your boot media.

The collectors that actually earn their keep on a Pi: cpu, meminfo, diskstats, filesystem, netdev, and thermal_zone. The thermal collector reads directly from /sys/class/thermal/thermal_zone*/temp — verify it's wired up before you trust your Grafana panel by hitting the metrics endpoint directly:

curl -s localhost:9100/metrics | grep node_thermal

# expect something like:
# node_thermal_zone_temp{type="cpu-thermal",zone="0"} 52000
# value is in millidegrees — divide by 1000 for Celsius in your PromQL

If that line is missing entirely, thermal_zone is either disabled or /sys/class/thermal is empty — check ls /sys/class/thermal/ on the host. On Pi 4 and 5 with a stock Raspberry Pi OS kernel, it's populated. If you're running a custom kernel or a container OS, it may not be.

Collectors worth explicitly disabling:

--no-collector.mdadm: Probes RAID arrays. There are none. It still runs and spits errors into the log on every scrape.
--no-collector.nfs and --no-collector.nfsd: Same pattern — reads from /proc/net/rpc/nfs which either doesn't exist or returns zeros, but the read happens on every scrape interval and counts as an I/O op on your SD card.
--no-collector.xfs and --no-collector.zfs: Unless you're running ZFS on your Pi (you're not), drop them.

The textfile collector is underused and worth knowing. Instead of enabling a heavyweight collector that runs on every Prometheus scrape, write a cron job that dumps a .prom file on a slower schedule — say, every 5 minutes — and let textfile serve it statically. This is how I expose custom metrics like UPS battery level and Docker container counts without polling on every 15-second scrape. The setup is two lines: point node_exporter at the directory with --collector.textfile.directory=/var/lib/node_exporter/textfile, then write a script that outputs valid OpenMetrics format and drops the result atomically:

#!/bin/bash
# write to tmp then mv — prevents Prometheus scraping a partial file
TMPFILE=$(mktemp)
echo "# HELP my_sensor_temp_celsius Temperature from I2C sensor" > "$TMPFILE"
echo "# TYPE my_sensor_temp_celsius gauge" >> "$TMPFILE"
echo "my_sensor_temp_celsius $(read_sensor.py)" >> "$TMPFILE"
mv "$TMPFILE" /var/lib/node_exporter/textfile/sensors.prom

The hwmon collector is worth a separate callout because the behavior differs by Pi model. On Pi 5, node_hwmon_temp_celsius appears in metrics, but the chip and sensor labels are sometimes empty strings, which means your Grafana query returns a result but the legend is blank and filtering by label breaks. Debug it before building any panel:

curl -s localhost:9100/metrics | grep hwmon

# Pi 5 — you might see:
# node_hwmon_temp_celsius{chip="platform_rp1_adc",sensor="temp1"} 42.651
# or chip="" sensor="" depending on kernel driver binding

If the labels are empty, use thermal_zone metrics instead — they're more reliable across Pi generations and the type label (cpu-thermal) is always populated.

Run node_exporter directly on the host, not in a container. This is the one place where "just containerize everything" actively hurts you. The filesystem collector needs to see real mount points, not the container's namespaced view. The netdev collector needs the host network namespace to report eth0 and wlan0 accurately. If you insist on containerizing it, you need --path.rootfs=/host, a bind mount of / into the container, host network mode, and PID namespace sharing — at which point you've added four failure surfaces to get the same data you'd get from a single binary on the host. On my own setup I run node_exporter as a systemd service on the Pi itself and only Prometheus and Grafana live in Docker. The service file is minimal:

[Unit]
Description=node_exporter
After=network.target

[Service]
User=node_exporter
ExecStart=/usr/local/bin/node_exporter \
  --no-collector.mdadm \
  --no-collector.nfs \
  --no-collector.nfsd \
  --no-collector.xfs \
  --no-collector.zfs \
  --collector.textfile.directory=/var/lib/node_exporter/textfile
Restart=on-failure

[Install]
WantedBy=multi-user.target

Drop that in /etc/systemd/system/node_exporter.service, run systemctl daemon-reload && systemctl enable --now node_exporter, and you get accurate host metrics with no namespace gymnastics.

Building the Dashboard: Panels That Catch Real Problems

The throttle ratio panel is the single most useful thing you can build for a Pi dashboard, and almost nobody sets it up. The metric node_cpu_scaling_frequency_hertz tells you the current CPU frequency; node_cpu_scaling_frequency_max_hertz tells you what it could be running at. The ratio between them is what actually matters:

node_cpu_scaling_frequency_hertz{cpu="cpu0"}
  /
node_cpu_scaling_frequency_max_hertz{cpu="cpu0"}

A healthy Pi sits at 1.0. When it drops to 0.6 or below under load, that's not a busy system — that's a thermally limited one. The distinction matters enormously for diagnosis: if you're seeing slow response times and the CPU panel shows 80% utilization, you'll assume the workload is the problem. But if the throttle ratio is simultaneously showing 0.5, the actual problem is heat, and adding more RAM or tuning your app won't fix it. Add a stat panel with thresholds: green above 0.9, orange between 0.7–0.9, red below 0.7. Pair it with a case without airflow and you'll see red within minutes of a heavy encode or compile job.

For memory, skip node_memory_MemFree_bytes entirely. Free memory on Linux is a misleading number because the kernel aggressively uses spare RAM for disk cache. The metric that actually tells you the system is approaching an OOM kill is node_memory_MemAvailable_bytes — that's the estimate of how much memory can be reclaimed quickly. Express it as a percentage of total RAM and set a Grafana alert threshold at 15%:

100 * node_memory_MemAvailable_bytes
      / node_memory_MemTotal_bytes

On a Pi 4 with 4GB running Prometheus, node_exporter, and a couple of containers, available memory can drift below 20% during scrape cycles without anything obviously wrong. That 15% threshold gives you a warning before the kernel starts killing processes. Wire this to a Grafana alert that fires to a webhook or ntfy.sh notification — you want to know about it before you SSH in to find your database process missing.

The disk I/O wait panel is what separates "my Pi is slow" from "my SD card is dying." Use rate(node_disk_io_time_seconds_total[2m]) on your root device — typically mmcblk0. This metric is the fraction of time the device is busy, so values above 0.8 sustained over several minutes mean processes are sitting in uninterruptible sleep waiting for the card. If you're running Prometheus's TSDB on the same SD card as the OS, you will see this spike every 2 hours when Prometheus compacts blocks to disk. That's expected. What's not expected is a sustained baseline above 0.4 with light workloads — that usually means the card is worn and write latency has degraded badly. Swap to a USB-attached SSD and watch the metric drop to near zero at idle.

Once you have panels worth keeping, export the dashboard JSON from Grafana's UI (Dashboard → Share → Export → Save to file) and commit it to your provisioning folder. The directory structure Grafana expects is /etc/grafana/provisioning/dashboards/, and you need exactly two files: a provider YAML and the dashboard JSON itself. The provider YAML is minimal:

# /etc/grafana/provisioning/dashboards/pi.yaml
apiVersion: 1
providers:
  - name: pi-dashboards
    type: file
    options:
      path: /etc/grafana/provisioning/dashboards

Drop the exported JSON in the same directory and restart Grafana. On next boot — or after a container recreate — the dashboard is there automatically, with no clicking through the UI. The one gotcha: Grafana will treat provisioned dashboards as read-only in the UI by default. If you need to make edits, either set allowUiUpdates: true in the provider YAML (and remember to re-export after changes), or edit the JSON directly and restart. Either way, keeping the JSON in version control means your entire dashboard survives a full wipe and reinstall in about 30 seconds.

Alerting Without AlertManager Complexity

The standard Prometheus alerting story involves deploying AlertManager as a separate container, writing a routing tree in YAML, configuring inhibition rules, and maintaining receiver configs that drift out of sync with your actual intent. For a single-operator Pi setup, that's architectural overhead that buys you nothing. Grafana's unified alerting — available since Grafana 9, and now the default in Grafana 10+ — handles the entire pipeline inside the same process you're already running. One fewer container, one fewer config file to version, one fewer thing to restart when you change a threshold.

Contact points are where most people reach for email and immediately hit deliverability problems. A better pairing for self-hosted setups: either a self-hosted ntfy.sh instance or a Telegram bot webhook. Both are free, both survive the kind of flaky home network conditions where your SMTP relay might queue for ten minutes before admitting failure. The ntfy contact point in Grafana takes a URL and an optional token — that's it. For Telegram, you need a bot token from BotFather and your chat ID, then configure a webhook contact point with:

# Grafana contact point — Telegram webhook
URL: https://api.telegram.org/bot<YOUR_BOT_TOKEN>/sendMessage
HTTP Method: POST
Content-Type: application/json

# Message body (set in "Optional Webhook Settings")
{
  "chat_id": "<YOUR_CHAT_ID>",
  "text": "{{ len .Alerts.Firing }} alert(s) firing on Pi: {{ range .Alerts.Firing }}{{ .Labels.alertname }} {{ end }}"
}

Two alert rules justify the entire setup and should be configured before you consider anything else. First: CPU temperature above 80°C sustained for 2 minutes. The 2-minute window matters — without it, a brief spike during a compile job pages you at 2am for nothing. The PromQL is straightforward if you're scraping the thermal zone via the node exporter's textfile collector or a custom metric:

# Alert: Pi CPU overtemperature
# Condition: for 2m
node_thermal_zone_temp{type="cpu-thermal"} > 80

# If you're exposing temperature as a custom gauge named rpi_cpu_temp_celsius:
rpi_cpu_temp_celsius > 80

The second rule is more valuable: projecting disk exhaustion using predict_linear. This catches slow write leaks — logging misconfiguration, a runaway SQLite database, Docker layer accumulation — long before df -h looks worrying:

# Alert: disk will fill within 24 hours based on 4h trend
# Condition: for 10m (avoids alerting on brief write bursts)
predict_linear(
  node_filesystem_avail_bytes{mountpoint="/", fstype!="tmpfs"}[4h],
  86400   # seconds = 24 hours
) < 0

The silencing gotcha hits every first-time Grafana alerting user: when Prometheus restarts — during an upgrade, after a config reload, during a Pi reboot — Grafana receives no data for that metric and fires the alert with state No Data by default. This causes a false page every time you touch your stack. The fix is buried in the alert rule editor under "Configure no data and error handling": set No data to OK during initial setup and any planned maintenance window. Switch it back to Alerting once you're confident the scrape is stable. It's a per-rule setting, not global, which is the right design — your temperature alert probably should page on no data in production, but your predict_linear disk rule absolutely should not fire just because Prometheus was down for 90 seconds.

Gotchas That Will Cost You an Afternoon

The Prometheus OOM kill on a 2 GB Pi 4 is the first thing that will happen to you, and it will happen silently — the container just disappears and docker ps shows nothing running. The default chunks_head_series limit is effectively unbounded, so Prometheus happily grows its in-memory TSDB head until the kernel OOM killer intervenes. Adding --storage.tsdb.head-chunks-write-queue-size=0 and --query.max-samples=5000000 to your Prometheus flags helps cap runaway query memory, but neither flag fixes the actual problem: WAL writes to a slow SD card create backpressure that keeps more chunks in memory longer than they should be. The real fix is pointing --storage.tsdb.path at a USB SSD. On a Pi 4 with a decent SSD attached over USB 3.0, WAL flush latency drops from hundreds of milliseconds to single digits, and Prometheus memory behavior becomes predictable.

# docker-compose.yml snippet — Prometheus targeting USB SSD mount
services:
  prometheus:
    image: prom/prometheus:v2.51.2
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus-data'
      - '--storage.tsdb.retention.time=15d'
      - '--storage.tsdb.head-chunks-write-queue-size=0'
      - '--query.max-samples=5000000'
    volumes:
      # USB SSD mounted at /mnt/usb on the host
      - /mnt/usb/prometheus:/prometheus-data
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro

Grafana's SQLite locking errors are a different failure mode but the same root cause: SD card I/O latency. When two browser tabs hit different dashboards simultaneously, Grafana opens concurrent write transactions against grafana.db, and SQLite's file locking stalls long enough that Grafana's own timeout fires first. You'll see database is locked in the container logs and a spinner in the UI that never resolves. Moving the Grafana data directory to a tmpfs path like /tmp/grafana-data does make the locking errors stop — SQLite on RAM is fast enough — but you lose your dashboard config on every reboot. For anything you want to keep, put the volume on the USB drive and set GF_DATABASE_WAL=true in your environment block, which at least enables WAL mode and reduces lock contention.

services:
  grafana:
    image: grafana/grafana:10.4.2
    environment:
      - GF_DATABASE_WAL=true
      - GF_PATHS_DATA=/var/lib/grafana
    volumes:
      # Same USB SSD, different subdirectory
      - /mnt/usb/grafana:/var/lib/grafana

Time sync problems produce some of the most confusing Prometheus behavior: graphs with gaps, negative rate calculations, and counters that appear to reset. On Raspberry Pi OS Lite, systemd-timesyncd is enabled by default, but if you've also installed ntp or chrony at any point, you can end up with two daemons fighting over the clock. Run timedatectl status and look at the NTP service line — if it says active but timedatectl timesync-status shows stale sync timestamps, something is blocking timesyncd. The clean fix is picking one daemon and fully disabling the other: sudo systemctl disable --now ntp followed by sudo systemctl enable --now systemd-timesyncd. Prometheus itself timestamps scraped metrics at the moment of the HTTP response, so even a 30-second clock jump introduces gaps that look like node failures in your dashboards.

ARM64 image availability is the gotcha that bites you late, after you've built out half your stack. prom/prometheus, prom/node-exporter, and grafana/grafana all publish proper linux/arm64 manifests and the images run natively on Pi 4 and Pi 5 without emulation. The problem is community exporters — things like MySQL exporters from third-party registries, or Homebridge metric plugins, or anything that hasn't seen a release in 18 months. Before adding any exporter to your docker-compose.yml, run:

docker manifest inspect prom/mysqld-exporter:v0.15.1 | grep architecture
# Expected output includes:
#   "architecture": "arm64"
# If you only see "amd64", you're running under QEMU emulation
# which means 3-5x slower scrape handling and potential segfaults

QEMU emulation on Docker Desktop or via binfmt_misc on the Pi itself won't crash immediately — the container starts fine, metrics appear to scrape, and everything looks normal until load increases or you hit a specific syscall that QEMU emulates incorrectly. Check the architecture before you wire something into your alert rules and depend on it being accurate.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

Free Self-Hosted Alternatives to Datadog: What Actually Works on Your Own Hardware

우병수 — Wed, 22 Jul 2026 08:10:57 +0000

TL;DR: Datadog's per-host pricing is deliberately structured to feel cheap at two or three nodes and expensive by the time you actually need it. The base infrastructure plan charges per host, per month, and that cost multiplies with every add-on — APM, log management, synthetics — each

📖 Reading time: ~21 min

What's in this article

Why Datadog's Pricing Forces the Self-Hosting Conversation
The Five Contenders and Their Honest Resource Costs
Setting Up Prometheus + Grafana in Docker Compose
VictoriaMetrics as a Prometheus Drop-In: Where It Wins
Netdata for Operators Who Want Zero Configuration
Matching Tool to Situation: The Decision Framework
Common Failure Modes and How to Catch Them Early

Why Datadog's Pricing Forces the Self-Hosting Conversation

Datadog's per-host pricing is deliberately structured to feel cheap at two or three nodes and expensive by the time you actually need it. The base infrastructure plan charges per host, per month, and that cost multiplies with every add-on — APM, log management, synthetics — each billed separately. Once you're past a handful of nodes, the invoice scales faster than your infrastructure does. The free tier lets you evaluate the product convincingly, then cuts retention to one day and limits custom metrics aggressively enough that real alerting becomes impossible without upgrading.

The specific operator problem isn't abstract cost anxiety — it's that you need continuous visibility into memory pressure, disk saturation rates, container restart loops, and traffic anomalies across your own hardware, and you don't want that telemetry leaving your network. Sending host-level metrics to a third-party SaaS means your capacity patterns, failure modes, and workload fingerprints are visible to another company's infrastructure. For homelab operators running LLM workloads or anything with sensitive data in memory, that's a hard no before you even look at the bill.

What self-hosted monitoring actually demands breaks down into three functional pieces that you can run as a monolith or compose from separate tools:

A metrics store with sustainable retention — something that handles 30-day or 90-day TSDB retention without OOMing on a modest VM. Prometheus with its default in-memory index is fine at small cardinality but will eat RAM if you're scraping hundreds of containers with high-label-cardinality metrics. VictoriaMetrics handles the same workload with a much smaller heap.
A dashboard layer — Grafana is the obvious answer here, and it talks to nearly every backend. The question is whether you want dashboards bundled with the metrics store (Netdata does this) or composable (Prometheus + Grafana).
An alerting engine — Prometheus AlertManager is the standard, but it's configuration-heavy. Tools like Grafana Alerting or Netdata's built-in alerting give you something usable without writing YAML state machines from scratch.

The composable route gives you more control and survives tool swaps. The monolith route (a single tool that does all three) gets you to alerting faster but makes migration painful later. Neither is universally correct — it depends on whether you're instrumenting three personal servers or managing a dozen-node homelab with mixed workloads. For readers also evaluating AI-assisted dev tooling in their stack, see our guide on AI Coding Tools in 2026: Cloud Copilots vs Local Models.

The Five Contenders and Their Honest Resource Costs

The most common trap when replacing Datadog is reaching for Prometheus + Grafana immediately because every tutorial points there — without accounting for what that stack actually costs at rest. Prometheus memory usage is governed almost entirely by cardinality: the number of unique label combinations across all your metrics. A modest home lab scraping 10 exporters with aggressive labeling (pod name, container name, instance, job, environment, region) can push RAM well above 1GB before you've connected a single dashboard. On my 32GB workstation this is invisible, but on a 4GB VPS it's the first thing that bites. Budget 500MB as an absolute floor for a minimal Prometheus instance and expect 1.5–2GB once you add node_exporter, cAdvisor, and a few application exporters with non-trivial label sets.

VictoriaMetrics solves exactly that problem. The single-node binary (victoria-metrics) is a drop-in Prometheus remote_write target and also speaks PromQL natively, so your existing Grafana dashboards require zero changes. The compression ratio compared to Prometheus's TSDB is genuinely dramatic — the same time series that chews through 10GB of disk in Prometheus will often land under 3GB in VictoriaMetrics. More importantly, its memory ceiling is roughly proportional to active query complexity, not raw cardinality, which means you can throw high-cardinality Kubernetes label sets at it without triggering OOM kills. It ships with its own alerting engine (vmalert) so you can drop Alertmanager from the stack entirely if you want a simpler footprint.

Netdata occupies a completely different design space. Rather than a scrape-then-store model, it runs as a streaming agent with 1-second resolution out of the box, auto-discovers Docker containers, systemd services, and dozens of application plugins without any configuration. The agent itself requires no cloud account — the Netdata Cloud UI is optional and the local dashboard runs fully offline at http://localhost:19999. The trade-off is that local retention defaults to roughly 1 day of per-second data (configurable via the dbengine settings in netdata.conf), so Netdata excels at real-time visibility and short-term debugging but isn't the right fit if you need 90-day trend analysis. RAM footprint on a typical node runs 80–150MB, which makes it genuinely viable on a Raspberry Pi 4.

Checkmk Raw Edition comes from a different heritage entirely — enterprise infrastructure monitoring rather than cloud-native metrics pipelines. The free tier has no host limit, which is unusual and worth taking seriously. The install footprint is heavier: it runs its own Apache instance, a Python-based check engine, and an OMD (Open Monitoring Distribution) site structure that isolates itself from your system packages. Expect 300–500MB RAM at idle for a single site. What you get in return is strong host state monitoring — service checks, hardware health via SNMP, Windows agent support — that Prometheus + Grafana handles awkwardly through a patchwork of exporters. If your environment mixes Linux servers, network gear, and Windows machines and you care more about "is this thing up and healthy" than "show me p99 latency histograms," Checkmk fits better than the metrics-first stack.

Here's how they compare on the dimensions that actually matter for small self-hosters:

Tool

Storage Backend

Min RAM (realistic)

Retention Default

Docker Support

Alerting Built-in

Biggest Dealbreaker for Small Self-Hosters

Prometheus + Grafana

Custom TSDB (local)

500MB–2GB

15 days

Via cAdvisor exporter

Alertmanager (separate process)

Cardinality spikes OOM without warning; five separate processes to maintain

VictoriaMetrics

Custom TSDB (local)

200–400MB

1 month (configurable)

Via cAdvisor or Prometheus exporters

vmalert (built-in)

No native dashboards — still need Grafana for visualization

Netdata

dbengine (local, tiered)

80–150MB

~1 day per-second, longer at lower resolution

Auto-discovers without config

Yes (health.d rules)

Long-term retention requires significant disk + tuning; PromQL not supported natively

Checkmk Raw

RRDtool

300–500MB

2 years (RRD, fixed-size, downsampled)

Agent or SNMP; container discovery not as automatic

Yes (native, rule-based)

RRDtool pre-aggregates data — you cannot query raw samples after the fact

The Checkmk RRDtool point deserves emphasis: RRDtool is a fixed-size circular buffer that downsamples older data automatically. After a week, your 1-minute resolution data gets consolidated into 5-minute averages and the originals are gone. For host-state monitoring this is fine. For post-incident forensics where you need to replay exactly what a metric did at 3:47 AM two weeks ago, it's a hard wall. Prometheus, VictoriaMetrics, and Netdata's dbengine all retain raw samples, which is the correct default for anything beyond basic uptime checks.

Setting Up Prometheus + Grafana in Docker Compose

The stack that trips up most first-timers isn't Prometheus itself — it's the combination of volume mount paths that silently reset on container restart, Grafana's security defaults that give you a blank login page with zero log output, and a prometheus.yml that scrapes nothing because the service name doesn't match what you think it does. Start with the compose file and get all three services talking before touching dashboards.

# docker-compose.yml
version: "3.8"

services:
  prometheus:
    image: prom/prometheus:v2.52.0
    container_name: prometheus
    restart: unless-stopped
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--storage.tsdb.retention.time=30d"
      - "--storage.tsdb.retention.size=10GB"
      - "--web.enable-lifecycle"   # lets you POST /-/reload without restarting

  grafana:
    image: grafana/grafana:10.4.2
    container_name: grafana
    restart: unless-stopped
    ports:
      - "3000:3000"
    volumes:
      - grafana_data:/var/lib/grafana
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
      - GF_SECURITY_ADMIN_PASSWORD=changeme   # still sets the admin account
    depends_on:
      - prometheus

  node_exporter:
    image: prom/node-exporter:v1.8.0
    container_name: node_exporter
    restart: unless-stopped
    ports:
      - "9100:9100"
    volumes:
      # read-only host mounts so node_exporter sees real filesystem/proc data
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - "--path.procfs=/host/proc"
      - "--path.sysfs=/host/sys"
      - "--path.rootfs=/rootfs"
      - "--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)"

volumes:
  prometheus_data:
  grafana_data:

# prometheus/prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "node_exporter"
    # use the Docker service name — not localhost, not the host IP
    static_configs:
      - targets: ["node_exporter:9100"]

The GF_AUTH_ANONYMOUS_ENABLED=true env var is the non-obvious one. Grafana 10.x ships with anonymous access off by default, and when you hit port 3000 on first load without it, you get a login form — which is expected — but if you're reverse-proxying through nginx or Tailscale, the redirect chain fails silently and you see a blank page or a 302 loop. Setting GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer alongside it means unauthenticated users land on dashboards read-only, which is fine for a home lab or internal tool. If you want to lock it down later, flip the env var and bounce the container — no data loss since everything lives in the named volume.

For Alertmanager, the routing tree is where most people create their own alert storms. The defaults are aggressive. Here's a skeleton that actually behaves:

# alertmanager/alertmanager.yml
global:
  resolve_timeout: 5m

route:
  group_by: ["alertname", "job"]
  group_wait: 30s        # wait this long before sending the first alert in a group
  group_interval: 5m     # how long to wait before sending new alerts in an existing group
  repeat_interval: 4h    # don't re-fire a still-active alert more often than this
  receiver: "slack-ops"

receivers:
  - name: "slack-ops"
    slack_configs:
      - api_url: "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
        channel: "#alerts"
        send_resolved: true
        title: '{{ template "slack.default.title" . }}'
        text: >-
          {{ range .Alerts }}
            *Alert:* {{ .Annotations.summary }}
            *Details:* {{ range .Labels.SortedPairs }} {{ .Name }}={{ .Value }} {{ end }}
          {{ end }}

inhibit_rules:
  - source_match:
      severity: "critical"
    target_match:
      severity: "warning"
    equal: ["alertname", "job"]   # suppress warning if critical is already firing

The group_wait: 30s / group_interval: 5m combination is what saves you from getting 40 Slack messages during a single node bounce. group_wait batches the initial burst; group_interval controls how long Alertmanager waits before sending updates to an already-notified group. Set repeat_interval below 1h and you will regret it during any multi-hour incident.

On storage: setting only --storage.tsdb.retention.time=30d means Prometheus will happily consume unbounded disk if your scrape cardinality spikes — a misbehaving exporter dumping thousands of label combinations will fill a volume in hours with no obvious warning until the container OOMs or the disk hits 100%. Set --storage.tsdb.retention.size=10GB alongside the time flag and Prometheus enforces whichever limit triggers first. On a typical single-host setup scraping node_exporter plus a few app exporters at 15s intervals, 10GB covers well over 30 days — but if you add Kubernetes pod metrics or high-cardinality app labels, that number compresses fast. Run prometheus_tsdb_head_series as a metric to watch cardinality in real time; anything above 500k active series on a single Prometheus instance starts hurting query latency noticeably.

VictoriaMetrics as a Prometheus Drop-In: Where It Wins

The single-binary story is genuinely what gets you to try it. One Docker command and you have a running TSDB that accepts Prometheus remote_write without touching your scrape configs:

docker run -d \
  --name victoria-metrics \
  -v /path/victoria-metrics-data:/victoria-metrics-data \
  -p 8428:8428 \
  victoriametrics/victoria-metrics:v1.101.0 \
  --retentionPeriod=6  # months, not days

Then in your existing prometheus.yml:

remote_write:
  - url: http://your-vm-host:8428/api/v1/write
    # keep your existing Prometheus scraping — VM just receives the data

High-cardinality label sets that were making Prometheus grind — think per-pod labels with UUID suffixes — compress dramatically under VictoriaMetrics's storage engine. You don't have to tune anything; the headaches just shrink.

The disk and memory difference is real and consistent enough to plan around. VictoriaMetrics's custom compression achieves 3–7x less disk usage than Prometheus's TSDB on comparable datasets — this is documented behavior from the project, reproducible by pointing both at the same remote_write stream and watching du -sh over a week. More practically: Prometheus holds its entire index in RAM. VictoriaMetrics doesn't need to, so its memory ceiling is lower under the same cardinality. On a constrained host — a small VPS or a shared homelab node — that difference determines whether the process survives a traffic spike.

VMAlert handles alerting rules and the compatibility story is mostly painless. Your existing Prometheus alerting rule YAML runs without modification in the common case. You point VMAlert at your VictoriaMetrics datasource, wire it to a notifier (Alertmanager if you have it, or a direct webhook if you don't), and you're done. For setups where you don't want to run a full Alertmanager stack, VMAlert can call a webhook directly — enough to cover Slack, PagerDuty, or whatever HTTP endpoint you're using for notifications.

# vmalert minimal config pointing at VM + direct webhook notifier
./vmalert \
  --datasource.url=http://localhost:8428 \
  --remoteWrite.url=http://localhost:8428 \
  --notifier.url=http://your-webhook-endpoint/alert \
  --rule=/etc/vmalert/rules/*.yml

Where VictoriaMetrics bites you: its PromQL dialect diverges just enough to break some Grafana dashboards built around Prometheus-specific functions. histogram_quantile with native histograms, certain label_replace edge cases, and subquery syntax can behave differently or fail silently — returning empty results rather than an error, which is the worst kind of failure for a monitoring dashboard. Before migrating any production dashboards, run both Prometheus and VictoriaMetrics in parallel on the same data stream, load every dashboard against both datasources, and diff the panel outputs. The incompatibilities are rarely showstoppers but they're also rarely obvious until a dashboard goes blank during an incident.

Netdata for Operators Who Want Zero Configuration

Most monitoring tools front-load the configuration burden — you spend an hour writing scrapers before seeing a single graph. Netdata inverts that. Run this on a bare Linux host and you'll have a full dashboard in under a minute:

wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh && sh /tmp/netdata-kickstart.sh

That single command installs the agent, detects your OS, and starts streaming CPU per-core, memory pressure, disk I/O, network throughput, and swap at 1-second granularity. No YAML, no scrape targets, no exporters. The dashboard is live at http://your-host:19999 before you've had time to open the docs. For operators who've wrestled with Prometheus + Grafana setup just to watch a box — this contrast is jarring in a good way.

Docker support is equally low-friction. The agent reads /var/run/docker.sock and automatically surfaces per-container cgroup metrics — CPU, memory limits vs. actual usage, network I/O — without requiring any label annotations or compose file edits. On my workstation running a dozen containers through Docker, every one of them showed up named and graphed within seconds of the agent starting. The one gotcha: the agent process needs socket access, so if you're running Netdata itself in a container you need to mount the socket explicitly. Running it on the host directly sidesteps the whole problem.

The storage behavior trips up most first-time users. Out of the box, Netdata stores metrics in a RAM ring-buffer — roughly one hour of history, gone on restart. For lab use that's fine; for anything production-adjacent it's a problem. The fix is switching to dbengine mode, which persists to disk with a configurable cap. Edit /etc/netdata/netdata.conf:

[db]
    mode = dbengine
    # storage tiers control how long data is kept at each resolution
    storage tiers = 3
    # tier 0: 1-second granularity, cap in MiB
    dbengine multihost disk space MB = 1024

With mode = dbengine and a 1 GB disk cap you get several days of 1-second data for a moderately busy host. The tiering system downsamples older data automatically — tier 1 goes to per-minute aggregates, tier 2 to hourly — so you're not burning disk storing raw second-level data indefinitely. This is in the docs but buried several pages deep, which is why a lot of people assume Netdata just doesn't do persistence.

The alert system is where the tool shows its seams. Rules live in /etc/netdata/health.d/*.conf and the format is Netdata-specific — not compatible with Prometheus alerting rules and not transferable if you ever switch stacks. Simple threshold alerts are genuinely readable:

 alarm: disk_space_usage
    on: disk.space
lookup: average -10m percentage of used
 units: %
 every: 1m
  warn: $this > 80
  crit: $this > 90
  info: disk space usage on $label:mount_point

For that kind of alert — one metric, one threshold, notify when breached — the format is cleaner than writing a PromQL expression. But if you need composite conditions (alert when CPU is high AND disk I/O is saturated AND available memory is below X), the rule language gets awkward fast. You can reference other alarms and use $this substitution, but there's no join-like construct across different collectors the way PromQL handles it natively. If your alerting needs are threshold-based on individual metrics, Netdata's built-in rules are genuinely usable. If you're building multi-signal correlation alerts, you'll hit the ceiling quickly and probably end up routing to an external system anyway.

Matching Tool to Situation: The Decision Framework

Most people spend hours comparing dashboards before asking the question that actually narrows the field: how many nodes, and how long does the data need to live? Those two numbers eliminate more options than any feature matrix.

Under 10 nodes — Netdata with dbengine, setup under 30 minutes

Netdata's dbengine mode stores metrics compressed on disk rather than purely in RAM, so you're not trading retention for node count at small scale. The install is a single command, the auto-detection of processes, containers, and network interfaces works without touching a config file, and the built-in dashboards are genuinely useful out of the box — no Grafana provisioning, no data source wiring. The trade-off: Netdata's alerting is passable but not composable. If you need routing rules that send disk alerts to one channel and app latency to another, you'll hit the ceiling fast. For a handful of hosts where you want eyes on the system in under an hour, nothing else comes close on setup time.

# Netdata one-liner install — works on Debian, Ubuntu, RHEL, Fedora
wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh
sh /tmp/netdata-kickstart.sh --stable-channel --disable-telemetry

# Confirm dbengine is active (default since v1.23)
grep -i dbengine /etc/netdata/netdata.conf
# Expected: [db] mode = dbengine

10–50 nodes, 90-day retention, already on Grafana: VictoriaMetrics single-node

VictoriaMetrics single-node binary accepts Prometheus remote_write from any scrape agent, compresses metrics better than stock Prometheus on disk, and runs with a fraction of the memory overhead at this node count. You drop the existing Grafana dashboards on top, point the data source at the VictoriaMetrics HTTP port, and you're done. There's no cluster mode to configure, no separate compaction job to monitor, and retention is a single startup flag: -retentionPeriod=3 for three months. The ops burden is genuinely low — it restarts cleanly, the WAL handles ungraceful shutdowns, and you don't need a sidecar to manage block lifecycle. Where it doesn't help: if your org already has Thanos running or you need multi-tenant query isolation, you're better off staying in the Prometheus ecosystem and not bifurcating your storage layer.

# Minimal VictoriaMetrics single-node with 90-day retention
docker run -d \
  --name victoriametrics \
  -p 8428:8428 \
  -v /data/vm:/storage \
  victoriametrics/victoria-metrics:v1.101.0 \
  -storageDataPath=/storage \
  -retentionPeriod=3  # months

# Prometheus scrape agent remote_write config
remote_write:
  - url: http://victoriametrics:8428/api/v1/write
    queue_config:
      max_samples_per_send: 10000

Multi-service stack needing composite alerting: Prometheus + Alertmanager + Grafana

The setup overhead is real — you're writing scrape configs, wiring Alertmanager receivers, tuning recording rules — but the ecosystem payoff is also real. Every non-trivial open source project ships a Prometheus exporter. Recording rules let you pre-compute expensive aggregations at scrape time so dashboards stay fast. Alertmanager's routing tree handles deduplication, grouping, and inhibition in ways that YAML-configured alternatives simply don't match. And if you ever need to scale retention, the Thanos sidecar drops onto an existing Prometheus deployment without requiring you to migrate your metrics format or re-instrument your apps. Accept that the first weekend of config is genuinely fiddly. The compounding value of the exporter ecosystem is worth it once you pass a handful of distinct services.

Legacy infra, Windows hosts, SNMP, or service-state monitoring: Checkmk Raw Edition

Prometheus exporters for Windows exist but they're a second-class experience — the WMI exporter requires PowerShell provisioning, the cardinality explodes with IIS or SQL Server metrics, and you end up writing PromQL that nobody else on your team can read. Checkmk's agent installs as an MSI, auto-discovers services, and speaks the same check language across Linux, Windows, and SNMP devices. The Raw Edition (fully open source, no node limit beyond what your hardware handles) gives you service-state monitoring — is this Windows service running, is this SNMP interface up — which is categorically different from metric-threshold alerting. The install is heavier: it runs its own web interface, its own notification engine, and an OMD-based site structure that takes getting used to. That's the trade you make for a tool that actually handles the breadth of a mixed infrastructure without needing a different agent per platform.

Common Failure Modes and How to Catch Them Early

The failure that kills self-hosted monitoring setups fastest isn't a missing feature — it's Prometheus cardinality explosion. Add one exporter that stuffs a request ID, trace ID, or UUID into a label value and you've created a new time series per request. Prometheus holds all active series in memory, so this compounds fast: a moderately busy API endpoint with request IDs in labels can generate millions of unique series within hours. The OOM kill comes with no warning and no useful log message, just a dead process. The fix is to audit before you ship. Hit :9090/tsdb/status in a browser and look at the "Top 10 series by label name" table — it shows which labels are contributing the most cardinality. If you see anything unbounded at the top, fix the exporter config or relabeling rules before it reaches production. Specifically, use metric_relabel_configs in your scrape config to drop or hash the offending label:

metric_relabel_configs:
  - source_labels: [request_id]
    action: labeldrop
    # dropping unbounded label before TSDB ingestion
    # hashing instead of dropping: action: replace, target_label: request_id, replacement: "redacted"

Grafana data source drift is subtle enough that it doesn't announce itself — panels just go empty and the error message is usually something useless like "No data." This happens constantly after a VictoriaMetrics migration because VM and Prometheus have different data source UIDs in Grafana, and any dashboard that hardcoded a UID internally now points at nothing. The reliable fix is to stop letting Grafana manage data source configuration interactively. Provision it via YAML with an explicit, stable UID:

# /etc/grafana/provisioning/datasources/victoriametrics.yaml
apiVersion: 1
datasources:
  - name: VictoriaMetrics
    type: prometheus
    uid: my-vm-ds-uid-001   # hardcoded — same across restores, migrations, upgrades
    url: http://victoriametrics:8428
    access: proxy
    isDefault: true

With that file in place, the UID survives container rebuilds and database restores. Any dashboard JSON that references my-vm-ds-uid-001 will resolve correctly regardless of what Grafana's internal state thinks. Without this, every environment rebuild risks silent dashboard rot.

Netdata's default alert thresholds are tuned for general-purpose Linux servers, which means they'll fire constantly on machines doing unusual workloads. Running Ollama with a large model loaded keeps GPU VRAM pinned near saturation by design — that's the point — but the memory pressure also shows up in system memory metrics and triggers memory.available alerts in a loop. The right fix isn't silencing all alerts, it's retuning specifically for that machine. Override the threshold in /etc/netdata/health.d/ram.conf (create it if it doesn't exist — overrides in /etc/netdata/health.d/ take precedence over /usr/lib/netdata/conf.d/):

# /etc/netdata/health.d/ram.conf
 alarm: ram_available
    on: system.ram
lookup: average -1m percentage of avail
 units: %
 every: 1m
  warn: $this < 5      # drop from default ~20% to 5% for inference workloads
  crit: $this < 2
  info: percentage of available RAM — tuned down for Ollama host

Checkmk's default active check interval looks reasonable on paper until you have 50+ hosts and start watching monitoring server CPU. Active checks — SSH-based checks, custom scripts, anything that spawns a process — stack up fast. The monitoring server is running one check process per host per service per minute. For non-critical services (disk space on a dev box, certificate expiry checks with 90-day certs), bump the check interval to 5 or 10 minutes in the service configuration. In Checkmk's Setup UI, that's under the "Normal check interval for service checks" rule in the ruleset browser, scoped to a host tag so you don't accidentally slow down critical path checks. The practical difference between a 1-minute and 5-minute interval for "disk usage on the NAS" is zero operational value — the difference in CPU headroom on a small monitoring host is measurable.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

Raspberry Pi Telemetry Dashboard for Home Automation: Setup to Production

우병수 — Mon, 20 Jul 2026 08:11:08 +0000

TL;DR: The sensor data problem isn't that you're not collecting it — Home Assistant's SQLite recorder captures almost everything by default. The problem is that SQLite starts choking on range queries once your history table grows past a few weeks of dense sensor data.

📖 Reading time: ~22 min

What's in this article

The Problem: Home Automation Data Is Scattered and Unactionable
Hardware and OS Prerequisites Before You Touch a Config File
Docker Compose Stack: InfluxDB 2.x, Grafana, and Telegraf
Pulling Home Automation Data Into the Pipeline
Building the Grafana Dashboard: Panels That Actually Tell You Something
Retention, Storage, and Keeping the Pi From Dying Under Load
Wiring Grafana Alerts Into Your Automation Pipeline

The Problem: Home Automation Data Is Scattered and Unactionable

The sensor data problem isn't that you're not collecting it — Home Assistant's SQLite recorder captures almost everything by default. The problem is that SQLite starts choking on range queries once your history table grows past a few weeks of dense sensor data. Run EXPLAIN QUERY PLAN SELECT * FROM states WHERE entity_id = 'sensor.living_room_temp' AND last_updated > datetime('now', '-30 days') against a mature HA database and you'll see full table scans. The default purge interval is 10 days precisely because the recorder isn't designed for long-term analytical queries — it's designed for state restoration and the last-24-hours view.

The fragmentation problem compounds this. A typical home automation setup ends up with Home Assistant for sensor state, a router admin panel (UniFi, OpenWrt, or whatever shipped with your ISP box) for network device presence and bandwidth, a UPS management interface for power draw, and maybe a separate Uptime Kuma instance for service health. None of these talk to each other. Correlating "my NAS went offline at 2am" with "there was a power fluctuation at 1:58am" requires you to mentally stitch together timestamps from four browser tabs. That's not analysis — that's archaeology.

The gap this stack actually closes is the distance between raw logged values and actionable pattern recognition. Having temperature readings in a log file tells you nothing until you can overlay them against HVAC runtime, time-of-day, and outdoor weather. Having power draw numbers is useless until you can set an alert threshold that fires when the whole-home draw spikes above baseline for more than five minutes. Grafana's alerting engine and InfluxDB's Flux query language are specifically built for exactly these cross-signal correlations — and they run comfortably on modest hardware.

Here's what this article builds, concretely: a Raspberry Pi 4 with 4GB RAM minimum (8GB gives you headroom to add more exporters later), running three containers via Docker Compose — InfluxDB 2.x as the time-series store, Grafana OSS (currently 10.x) as the visualization and alerting layer, and a lightweight Telegraf collector that pulls from Home Assistant's REST API, local system metrics, and SNMP or ping checks for network devices. The Pi handles this workload without breaking a sweat; InfluxDB's TSM storage engine is far more efficient than SQLite for time-series shapes, and Grafana's memory footprint at idle is well under 200MB. The whole stack fits on a 32GB SD card with room to spare, though a USB SSD is strongly recommended for write endurance on any InfluxDB deployment that's ingesting data continuously.

Hardware and OS Prerequisites Before You Touch a Config File

The SD card failure mode is the first thing to get right, and most tutorials skip it. InfluxDB 2.x is write-heavy by design — every scrape interval hammers the storage layer with small random writes, which is exactly what wears out NAND flash on cheap microSD cards. A Class 10 card will die quietly, usually corrupting your database before it throws any obvious errors. Before you commit to any storage layout, boot the Pi, run your planned scrape workload for 20 minutes, and check what's actually hitting the disk:

sudo apt install iotop -y
sudo iotop -ao --only

The -a flag shows accumulated I/O rather than instantaneous rates, and --only filters to processes actively doing I/O. If you see InfluxDB or the kernel's journal flushing multiple megabytes per minute, a microSD is going to lose that fight inside a few months of continuous operation. A USB 3.0-attached SSD — even a cheap SATA SSD in a USB enclosure — changes the write endurance math entirely. On a Pi 4, USB 3.0 is shared bandwidth with the NIC, so don't expect full SATA speeds, but you'll still get dramatically better random write performance than any SD card. On a Pi 5, the situation is better: you can attach an NVMe drive via the PCIe FFC connector with an appropriate HAT.

Raspberry Pi OS Lite (64-bit, Bookworm) is the correct base image — not the desktop variant, not the 32-bit build. InfluxDB 2.x only ships ARM64 binaries; the 32-bit ARMv7 Debian packages don't exist in their official repo. Bookworm (Debian 12) also gives you a kernel new enough to not fight with Docker's network namespace handling. Flash it with Raspberry Pi Imager, use the advanced options to pre-configure your SSH key and hostname before first boot, and never attach a monitor. If you're booting headless from the start, you eliminate an entire category of "works on my desk, breaks in the closet" problems.

Install Docker Engine using the official convenience script, not whatever version is in the Raspberry Pi OS package repos — that copy is usually multiple major versions behind:

# One-liner installs the current stable Docker Engine for your architecture
curl -fsSL https://get.docker.com | sh

# Add your user to the docker group so you're not sudo-ing every command
sudo usermod -aG docker $USER

# Log out and back in, then verify
docker run --rm hello-world

Don't install Docker Desktop — it adds a VM layer that makes zero sense on a Pi and its ARM builds have historically lagged. The Engine-only install is what you want for headless server use.

Assign a static IP through your router's DHCP reservation table using the Pi's MAC address, not by editing /etc/dhcpcd.conf on the Pi itself. The reason is operational: when you need to re-image the Pi — and you will need to, eventually — a static lease on the router survives the wipe. Your Grafana bookmarks, your MQTT broker address, your n8n webhook URLs all stay valid without touching any config. Hardcoding the IP in dhcpcd.conf means you have to remember to replicate that config every time you re-flash. The router-side reservation costs you nothing and makes the Pi functionally disposable, which is exactly the right operational posture for a device running 24/7 in a closet.

Docker Compose Stack: InfluxDB 2.x, Grafana, and Telegraf

The biggest mistake people make with this stack is using latest tags everywhere and wondering why things break after a docker compose pull three months later. Pin InfluxDB specifically — the 2.x config model, Flux query language behavior, and bucket auth scheme changed enough between minor versions that an unpinned update will silently break your Telegraf writes. Use influxdb:2.7 and telegraf:1.30. Grafana OSS moves faster and is generally safer to track at a recent pinned tag, but even there, pin it once your dashboards are stable.

# docker-compose.yml
version: "3.8"

networks:
  telemetry:
    driver: bridge

volumes:
  influxdb-data:
  grafana-data:

services:
  influxdb:
    image: influxdb:2.7
    container_name: influxdb
    restart: unless-stopped
    networks:
      - telemetry
    ports:
      - "8086:8086"
    volumes:
      - influxdb-data:/var/lib/influxdb2
    environment:
      - DOCKER_INFLUXDB_INIT_MODE=setup
      - DOCKER_INFLUXDB_INIT_USERNAME=${INFLUXDB_ADMIN_USER}
      - DOCKER_INFLUXDB_INIT_PASSWORD=${INFLUXDB_ADMIN_PASSWORD}
      - DOCKER_INFLUXDB_INIT_ORG=${INFLUXDB_ORG}
      - DOCKER_INFLUXDB_INIT_BUCKET=${INFLUXDB_BUCKET}
      # Token is loaded from .env — never hardcode here
      - DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=${INFLUXDB_ADMIN_TOKEN}

  grafana:
    image: grafana/grafana-oss:11.1.0
    container_name: grafana
    restart: unless-stopped
    networks:
      - telemetry
    ports:
      - "3000:3000"
    volumes:
      - grafana-data:/var/lib/grafana
    depends_on:
      - influxdb

  telegraf:
    image: telegraf:1.30
    container_name: telegraf
    restart: unless-stopped
    networks:
      - telemetry
    volumes:
      # Read-only bind mount — telegraf has no reason to write back to this file
      - ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
    depends_on:
      - influxdb
    # Pi-specific: expose host thermal zone to the container
    devices:
      - /dev/gpiomem:/dev/gpiomem

# .env  (chmod 600 this file, add to .gitignore)
INFLUXDB_ADMIN_USER=admin
INFLUXDB_ADMIN_PASSWORD=changeme_strong_password
INFLUXDB_ORG=homelab
INFLUXDB_BUCKET=telemetry
# Generate with: openssl rand -hex 32
INFLUXDB_ADMIN_TOKEN=your_generated_hex_token_here

DOCKER_INFLUXDB_INIT_MODE=setup only runs the bootstrap sequence when the data volume is empty. Once influxdb-data has content, InfluxDB ignores those env vars entirely — so leaving them in the compose file after first boot is harmless. What's not harmless: hardcoding the token in the compose file itself, which ends up in shell history, CI logs, and git repos. Keep it in .env, lock the permissions, and add .env to your .gitignore before the first git add.

# telegraf.conf — minimal but useful for a Pi telemetry node

[agent]
  interval = "10s"
  round_interval = true
  flush_interval = "10s"
  hostname = "raspberrypi"

[[outputs.influxdb_v2]]
  # Use the service name on the shared bridge — not localhost
  urls = ["http://influxdb:8086"]
  token = "${INFLUXDB_ADMIN_TOKEN}"
  org = "${INFLUXDB_ORG}"
  bucket = "${INFLUXDB_BUCKET}"

[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false

[[inputs.mem]]

[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "overlay"]

[[inputs.net]]
  interfaces = ["eth0", "wlan0"]

[[inputs.temp]]
  # On Pi 4/5 this reads from /sys/class/thermal/thermal_zone0/temp
  # Reports in Celsius as 'temp' measurement with tag sensor=cpu-thermal

Two things to get right in the Telegraf config that the docs don't emphasize: first, use the Docker service name influxdb as the hostname in urls, not localhost — these containers are on the same bridge network, and localhost inside the Telegraf container points to the Telegraf container itself. Second, pass the token and org as environment variables referencing the same .env values rather than duplicating the literal string. Telegraf 1.30 expands ${VAR} syntax in the config file directly, so you get one source of truth. The inputs.temp plugin on a Pi 4 or Pi 5 reads from /sys/class/thermal/ without any extra kernel modules — you'll see a measurement named temp with a sensor tag value of cpu-thermal, which maps cleanly to a Grafana stat panel.

Named Docker volumes for InfluxDB and Grafana data are non-negotiable if you ever run docker compose down — which you will, during updates. Bind mounts work too, but named volumes survive accidental directory moves and are easier to back up with docker run --rm -v influxdb-data:/data busybox tar czf - /data. The only bind mount in this stack is telegraf.conf mounted read-only, because you want to edit it from the host without exec-ing into the container, and Telegraf has no legitimate reason to modify its own config file at runtime. If it can't write there, a misconfiguration or a bad plugin can't corrupt it either.

Pulling Home Automation Data Into the Pipeline

The least obvious thing about the Home Assistant → InfluxDB path is that it's event-driven, not time-series sampled. That distinction will bite you the first time you build a Grafana graph and see a door sensor drop off the chart for six hours because nothing happened. Keep that in mind for everything that follows.

Home Assistant InfluxDB v2 Integration

The official integration supports InfluxDB v2's API but you have to opt into it explicitly — the default assumes v1. Your configuration.yaml block needs to look like this:

influxdb:
  api_version: 2
  host: 192.168.1.x        # your Pi's LAN IP, not localhost if HA runs elsewhere
  port: 8086
  ssl: false                # skip TLS unless you've got a cert chain sorted
  token: your-influxdb-token-here
  organization: homelab
  bucket: home_automation
  include:
    entity_globs:
      - sensor.living_room_*
      - sensor.outdoor_*
      - binary_sensor.front_door
  exclude:
    entity_globs:
      - sensor.*_last_updated
      - sensor.*_friendly_name

The include/exclude filtering is not optional if you care about bucket hygiene. A default Home Assistant install with a handful of integrations will push thousands of state changes per day — most of them are UI state, internal metadata, or automation flags that have no business in your metrics store. Be specific with entity_globs from day one; retrofitting this after six weeks of accumulated noise is painful. After restarting HA, tail the log and watch for InfluxDB write errors — the most common is a mismatched organization name (case-sensitive).

MQTT Bridge via Telegraf

Zigbee2MQTT, Tasmota, and ESPHome all publish to MQTT natively. Wiring Telegraf directly to Mosquitto means you're capturing that data independently of whether Home Assistant is running — useful during HA restarts or migrations. In your telegraf.conf:

[[inputs.mqtt_consumer]]
  servers = ["tcp://192.168.1.x:1883"]
  topics = ["home/#", "zigbee2mqtt/#", "tele/+/SENSOR"]
  data_format = "json"
  json_time_key = ""         # let Telegraf timestamp on ingest
  qos = 0
  connection_timeout = "30s"
  persistent_session = false
  client_id = "telegraf-home"

The data_format = "json" setting does most of the heavy lifting — Telegraf will flatten nested JSON keys into field names automatically. A Zigbee2MQTT payload like {"temperature": 21.4, "humidity": 58, "linkquality": 103} lands in InfluxDB as three separate fields on one measurement. Tasmota's tele/+/SENSOR payloads are slightly more nested, so you may need json_string_fields or a processor to unwrap them cleanly. The wildcard home/# topic is aggressive — add a tag filter or namepass rule if you're picking up MQTT traffic from non-sensor sources like presence detection or alarm states.

HTTP Scraping for Power Monitors

Shelly devices expose a JSON REST endpoint at http://<device-ip>/status (gen1) or http://<device-ip>/rpc/Switch.GetStatus?id=0 (gen2/Plus). TP-Link Kasa requires a slightly different approach since it uses an encrypted UDP protocol by default, but the python-kasa library can expose a local HTTP shim you can then scrape. Telegraf's inputs.http handles both cleanly:

[[inputs.http]]
  urls = [
    "http://192.168.1.50/status",    # Shelly 1PM
    "http://192.168.1.51/status",    # Shelly EM
  ]
  method = "GET"
  interval = "10s"
  data_format = "json"
  name_override = "shelly_power"

  # Pull only what matters
  json_query = "meters"

A 10-second interval is a reasonable floor for energy data on a Pi 4 — you get enough resolution to catch appliance startup spikes without the CPU overhead of sub-second polling across a dozen devices. Shelly gen1 devices have no rate limiting on the local HTTP endpoint so you can go tighter if needed, but energy billing math doesn't benefit from more than 10s granularity. One gotcha: if your Shelly is on a different VLAN from the Pi, IGMP snooping or inter-VLAN routing issues will silently drop the scrape without a useful error in Telegraf — check with curl from the Pi first before debugging the plugin config.

Handling the Gap Problem in Grafana

Because the HA InfluxDB integration only writes on state change, binary sensors — door contacts, motion detectors, window sensors — will appear as single data points with long gaps between them. Grafana interprets these gaps as missing data and draws nothing. The fix is per-panel: under Field overrides, set Graph styles → Fill below to → previous, or in the older panel editor look for Null values: connected and set it to Fill: previous. This tells Grafana to extend the last known value forward until the next data point, which is the semantically correct behavior for a door that's been closed since 9am. Do not apply this setting to numeric sensors like temperature — you want the gap to be visible there, because a missing temperature reading means your sensor is offline, not that the temperature is unchanged.

Building the Grafana Dashboard: Panels That Actually Tell You Something

Most Grafana dashboards I've seen in home-lab write-ups are tourist dashboards — pretty graphs that you look at once and never open again. The ones worth keeping are built around questions you actually ask: is the house using too much power right now?, did the bedroom get hot while I was away?, which sensor went silent three hours ago? Panel layout should answer those in order, top to bottom, without hunting.

Flux Over InfluxQL — Not a Preference, a Capability Gap

Connect Grafana to InfluxDB using the InfluxDB (Flux) datasource, not the legacy InfluxQL one. The config difference is a single radio button in the datasource settings, but the query capability gap is significant. Flux's aggregateWindow() handles time bucketing and gap-filling natively. movingAverage() smooths noisy sensor data without a subquery. A 5-minute average CPU temperature query on my setup looks like this:

from(bucket: "homelab")
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "temp" and r.host == "rpi-main")
  |> aggregateWindow(every: 5m, fn: mean, createEmpty: false)
  |> yield(name: "mean_temp")

createEmpty: false is the flag the docs bury — without it, gaps in sensor data produce null-filled windows that Grafana renders as dropped lines, which looks like sensor failures when it isn't. The yield() at the end is required when you're running multiple queries in the same panel; Grafana uses the yield name as the series label. For power draw with a moving average overlay, chain |> movingAverage(n: 3) after the aggregation window — that smooths over momentary spikes without hiding real sustained load.

Panel Layout Tied to Operator Workflow

The layout I run follows a deliberate information hierarchy. Top row: two single-stat panels — current power draw in watts and current indoor temperature. These use the Stat panel type with thresholds configured (green below 200W, yellow 200–350W, red above). They give you a pass/fail read at a glance without opening a graph. Middle section: two time-series panels spanning 24 hours — one for power, one for temperature across zones. Time range is fixed to now-24h rather than the dashboard-global variable so it stays useful regardless of what someone drags the global range to. Bottom row: a Table panel that queries last-seen timestamps for every MQTT device:

from(bucket: "homelab")
  |> range(start: -24h)
  |> filter(fn: (r) => r._measurement == "mqtt_last_seen")
  |> last()
  |> keep(columns: ["device_id", "_time", "_value"])
  |> sort(columns: ["_time"], desc: false)

Sorting ascending by _time puts the longest-silent device at the top — exactly the one you want to see first. Apply a cell color threshold on the _time column: anything older than 30 minutes turns red. That table has caught two dead sensor batteries on my setup before I'd have noticed any alert.

Alerting Without Giving Grafana Cloud Your Email Address

Grafana OSS 9+ has a fully functional alerting engine without any cloud dependency. Set up a contact point under Alerting → Contact points. For local email, configure SMTP in grafana.ini:

[smtp]
enabled = true
host = your-smtp-relay:587
user = alerts@yourdomain.local
password = yourpassword
from_address = grafana@yourdomain.local
skip_verify = false

For anything more than email — routing to a specific n8n flow, posting to a local Matrix room, triggering a Home Assistant webhook — use the Webhook contact point type with your n8n webhook URL. The payload Grafana sends is JSON with the alert state, labels, and values. In n8n, a single Webhook trigger node receives it, and you can branch from there. The alert rule that matters most: on the power draw panel, define an alert that fires when consumption exceeds your threshold for 5 consecutive minutes. That "for" duration is set in the alert rule's Pending period field — set it to 5m. Without a pending period, a kettle or microwave kicks off a false alert every morning.

Dashboard JSON in Version Control, Not Just in the Database

Grafana stores dashboards in its internal SQLite or Postgres database, which means a container rebuild or volume wipe takes them with it. The fix is provisioning: export the dashboard JSON from Dashboard settings → JSON model, commit it to a git repo, then mount it into the container and tell Grafana where to look. In your Docker Compose file:

services:
  grafana:
    image: grafana/grafana-oss:10.4.2
    volumes:
      - ./grafana/provisioning:/etc/grafana/provisioning
      - ./grafana/dashboards:/var/lib/grafana/dashboards
    environment:
      - GF_PATHS_PROVISIONING=/etc/grafana/provisioning

The provisioning directory needs a dashboard provider config at ./grafana/provisioning/dashboards/homelab.yaml:

apiVersion: 1
providers:
  - name: homelab
    folder: Home Automation
    type: file
    disableDeletion: true
    updateIntervalSeconds: 30
    options:
      path: /var/lib/grafana/dashboards

disableDeletion: true prevents someone from accidentally deleting the provisioned dashboard from the UI — it'll just reappear on the next sync interval. The updateIntervalSeconds: 30 means a git pull followed by copying the updated JSON into the dashboards directory takes effect in under a minute without a container restart. That's the loop: edit in Grafana UI, export JSON, commit, done. Container rebuilds are now safe.

Retention, Storage, and Keeping the Pi From Dying Under Load

The most common Pi telemetry setup failure I see documented online skips the storage math entirely and then wonders why the SD card is dead or the disk is full in three weeks. Before you commit to any hardware, run your intended Telegraf config for 24 hours and pull actual write rates from InfluxDB's built-in /metrics endpoint — it exposes Prometheus-format output at http://localhost:8086/metrics and includes influxdb_write_points_ok_total so you can calculate your actual ingestion rate rather than guessing. Pair that with docker stats watching the InfluxDB container and you'll know whether a 64GB SSD is sufficient before you're committed. For a typical home setup — a handful of temperature/humidity sensors, one or two power monitors, and basic system metrics — 30 days of retention on a 64GB SSD is about the practical ceiling before you start chasing disk space.

Set your bucket retention policy at init time, not after the fact. If you're running InfluxDB 2.x in Docker, pass it during the first boot:

docker run -d \
  --name influxdb \
  -p 8086:8086 \
  -v influxdb-data:/var/lib/influxdb2 \
  -e DOCKER_INFLUXDB_INIT_MODE=setup \
  -e DOCKER_INFLUXDB_INIT_USERNAME=admin \
  -e DOCKER_INFLUXDB_INIT_PASSWORD=yourpassword \
  -e DOCKER_INFLUXDB_INIT_ORG=homelab \
  -e DOCKER_INFLUXDB_INIT_BUCKET=telemetry \
  -e DOCKER_INFLUXDB_INIT_RETENTION=720h \  # 30 days
  influxdb:2.7

Changing retention after the fact via the UI works fine, but the old data doesn't retroactively shrink — InfluxDB runs shard compaction on its own schedule. If you overshoot your retention and need to reclaim space immediately, you'll need to drop and recreate the bucket, which means losing history. Setting it correctly upfront is the only safe path.

Telegraf input intervals are where cardinality problems quietly compound. The safe defaults: system-level metrics (CPU, memory, disk I/O) at interval = "10s", HTTP scrapes of external APIs or device endpoints at interval = "30s", and MQTT inputs left event-driven — meaning no polling interval, just [[inputs.mqtt_consumer]] subscribing and writing on receipt. The mistake is setting a global interval and letting MQTT inherit it. If a sensor fires 20 events per minute and you're also polling it on a 10-second schedule, you're doubling your write volume and creating duplicate tags that inflate cardinality. InfluxDB's cardinality limit isn't documented as a hard wall in the OSS tier, but series cardinality above roughly 10–15 million on a constrained device will cause measurable query slowdown.

On a Pi running from SD card, InfluxDB's WAL (write-ahead log) will kill your card faster than anything else in this stack. The WAL flushes frequently by design — it's what makes InfluxDB durable. After the first 48 hours of operation, run:

sudo dmesg | grep -i error

Any mmc0 I/O errors or EXT4-fs error lines at this stage mean your SD card is already degrading under the write load. On an SSD via USB 3.0 or the Pi 5's PCIe M.2 slot this doesn't happen — SSDs handle random write patterns from WAL workloads without issue. If you're committed to SD card for some reason, at minimum move the InfluxDB data directory to a USB drive and keep the OS on the card. Don't run the WAL on SD and expect it to last a year.

For backups, a nightly cron job to an NFS share or external USB drive is straightforward and the restore path is actually reliable — unlike some tools that produce backup archives you can't test without a full cluster. The InfluxDB CLI backup format is portable across 2.x minor versions:

# /etc/cron.d/influx-backup
0 2 * * * root influx backup /mnt/backup/influx/$(date +\%Y-\%m-\%d) \
  --host http://localhost:8086 \
  --token $INFLUX_TOKEN

Store INFLUX_TOKEN in a root-only readable file and source it in the cron environment, or pass it via a wrapper script — cron's environment doesn't inherit your shell exports. To verify the backup is actually restorable, spin up a temporary InfluxDB container, run influx restore against it, and confirm your bucket and series show up. Do this once. Backups you've never tested restored aren't backups.

Wiring Grafana Alerts Into Your Automation Pipeline

The most underrated part of a Raspberry Pi telemetry setup isn't the data collection — it's closing the feedback loop so the dashboard actually does something. Grafana's webhook contact point paired with an n8n trigger is how you get from "I can see the power draw spiking" to "the switch turned off automatically and I got a push notification." The wiring is straightforward once you know the exact shape of the payload.

In Grafana, go to Alerting → Contact points → New contact point, select Webhook, and paste in your n8n webhook URL. On my setup that looks like http://192.168.1.x:5678/webhook/grafana-alerts — internal IP, no auth header required since it's LAN-only, though you can add a Basic Auth header if your n8n instance is exposed. The payload Grafana sends looks roughly like this:

{
  "receiver": "n8n-webhook",
  "status": "firing",
  "alerts": [
    {
      "status": "firing",
      "labels": {
        "alertname": "HighPowerDraw",
        "device": "washing_machine_plug"
      },
      "values": {
        "watts": 2340
      },
      "state": "Alerting"
    }
  ]
}

Inside n8n, the Webhook trigger node receives that body and you route on {{ $json.alerts[0].status }} and {{ $json.alerts[0].labels.device }}. Drop an IF node after it: if status is firing and device matches the plug you care about, continue down the action branch. The HTTP Request node then calls Home Assistant's REST API:

POST http://homeassistant.local:8123/api/services/switch/turn_off
Headers:
  Authorization: Bearer YOUR_HA_LONG_LIVED_TOKEN
  Content-Type: application/json

Body:
{
  "entity_id": "switch.washing_machine_plug"
}

Chain a Pushover node after that (n8n has a native Pushover node — credentials take 30 seconds to configure) and template the message with {{ $json.alerts[0].values.watts }}W detected on {{ $json.alerts[0].labels.device }} — switch turned off. The whole flow from Grafana firing to your phone buzzing runs in under a minute of wall time once it's built, and maybe 10 minutes to actually build it. For a broader look at how webhook-driven pipelines like this fit into a self-hosted automation stack, see our guide on Workflow Automation in 2026: n8n, Zapier, and Self-Hosted Pipelines.

The gotcha that wastes real time: on Grafana versions below 10.x, the legacy alerting engine either doesn't send a resolved webhook at all, or sends it in a different schema than the firing payload. That means your n8n flow never gets the "all clear" signal, and if your logic depends on toggling state back — like re-enabling a switch once the draw drops — it silently breaks. Upgrade to Grafana 10.x or later; the unified alerting engine sends both firing and resolved states consistently, and the alerts[0].status field is reliable in both directions. If you're pinned to an older version for some reason, the workaround is polling Home Assistant's sensor state from n8n on a schedule and checking it independently — ugly, but functional.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

5 Lightweight Tailscale Alternatives Worth Running on Your Own Hardware

우병수 — Fri, 17 Jul 2026 08:10:19 +0000

TL;DR: Tailscale's WireGuard mesh is elegant, and the client software works well. The problem isn't the protocol — it's that the glue holding your mesh together lives on Tailscale's servers, behind closed-source code you can't audit or replicate.

📖 Reading time: ~17 min

What's in this article

Why Tailscale Becomes the Wrong Tool
Comparison Table: The Five Alternatives at a Glance
Headscale: Tailscale Clients, Your Control Server
Netbird: WireGuard Mesh With a Built-in Admin UI
Nebula: Flat Mesh Without Any Central Relay
Picking the Right Tool for Your Setup

Why Tailscale Becomes the Wrong Tool

The Coordination Server Is the Hidden Dependency

Tailscale's WireGuard mesh is elegant, and the client software works well. The problem isn't the protocol — it's that the glue holding your mesh together lives on Tailscale's servers, behind closed-source code you can't audit or replicate. Every device auth, every key rotation, every peer discovery event goes through login.tailscale.com. If that goes down, your mesh doesn't degrade gracefully — peers that haven't cached routes stop resolving each other entirely. For a homelab this is inconvenient. For anything you've wired production services through, it's a real outage you have zero ability to fix.

The DERP relay situation compounds this. WireGuard needs direct UDP paths between peers; when NAT traversal fails (CGNAT, strict firewalls, mobile carriers), Tailscale falls back to its DERP relay servers. You can run your own DERP node, but your traffic still gets routed through Tailscale's coordination layer to discover which relay to use. You're not actually air-gapped from their infrastructure — you've just moved one hop. Self-hosters running nodes behind carrier-grade NAT will hit DERP relay traffic constantly, and those relays are outside your control, your jurisdiction, and your incident-response loop.

The free tier's device limit is the friction point that usually forces the decision. The cap isn't enormous, and once you start counting — workstations, servers, VPS nodes, phones, a Raspberry Pi or two, a NAS — you hit it faster than expected. Bumping to a paid plan to host your own infrastructure feels backwards. The pricing isn't outrageous, but you're now paying a recurring fee for a control plane that still isn't yours, with no self-hosted alternative available at any price tier through Tailscale itself.

The air-gap question is where Tailscale fully stops being an option. Regulated environments, isolated home lab segments you want genuinely off the internet, or just paranoia about external dependencies — none of these are solvable with stock Tailscale. The coordination server must be reachable. There's no offline mode, no bundle-your-own-server path, no "here's the protocol spec, run it yourself." Headscale exists as a third-party reimplementation, which is a reasonable answer, but it's worth being clear that it's a reverse-engineered compatibility layer, not something Tailscale ships or supports.

The actual operator question is specific: which tool lets you own the full control plane — key exchange, peer discovery, relay infrastructure — without turning network administration into a part-time job? Running raw WireGuard with manual config is the maximalist ownership answer, but you're writing wg0.conf entries by hand and scripting your own key rotation. The sweet spot is a tool that gives you a self-hosted coordination server with a reasonable operational surface area: one or two containers, a config file you can read, and upgrade paths that don't require re-architecting your mesh. That's the gap the alternatives in this piece are trying to fill.

Comparison Table: The Five Alternatives at a Glance

The uncomfortable truth about all five of these tools: you're not escaping operational complexity, you're just moving it somewhere you control. Tailscale's SaaS model absorbs that complexity into their infrastructure. Every option below hands it back to you in the form of a control plane you have to run, upgrade, and back up. That trade is worth making — but go in clear-eyed about what you're taking on.

Tool

Control Plane

WireGuard-based

NAT Traversal

Min Server RAM

Biggest Gotcha

Headscale

Self-hosted

Yes

DERP relay (Tailscale's protocol, self-hostable)

~50 MB

Lags behind official Tailscale client feature releases; some client features silently break

Netbird

Self-hosted or vendor

Yes

STUN/TURN + ICE (standard WebRTC stack)

~150 MB (full stack: signal + relay + dashboard)

Self-hosted stack is four separate services; docker-compose drift between versions is real

Nebula

Self-hosted (CA + lighthouses)

No (custom UDP overlay)

Lighthouse-assisted hole-punching

~20 MB

Certificate rotation is fully manual; no built-in revocation workflow

ZeroTier (self-hosted controller)

Self-hosted

No (ZeroTier protocol over UDP)

Distributed roots + optional moons

~80 MB

Moon (custom root) propagation can take several minutes; clients occasionally ignore it

innernet

Self-hosted

Yes

None — requires routable server or manual port-forward

~30 MB

No NAT traversal at all; all peers need path to the innernet server

The NAT traversal column deserves more attention than it usually gets. Headscale inherits Tailscale's DERP relay infrastructure — you can self-host DERP servers, but most people don't, which means they're still depending on Tailscale's relay nodes when direct connections fail. Netbird's ICE/STUN/TURN stack is more operationally familiar if you've run WebRTC infrastructure, but TURN relay servers are bandwidth-intensive when connections can't punch through directly. Nebula and ZeroTier handle traversal at the protocol level without depending on a relay you separately maintain. innernet doesn't even try — it's the honest option that skips the magic and requires real network access to the server.

RAM footprints above reflect the server-side control plane only, not the per-node agent. Agent overhead on nodes is modest across all five — typically under 15 MB RSS once the tunnel is established. Where these numbers start to matter is when you're running the control plane on a small VPS (1 GB RAM, shared) alongside other services. Nebula's lighthouse at ~20 MB barely registers. Netbird's full self-hosted stack at ~150 MB means you'll want it on a dedicated instance or at minimum a 2 GB droplet with swap configured properly.

Operationally, the gotcha column is the one to read twice before committing. Headscale's lag behind upstream Tailscale client releases has bitten people trying to use newer Tailscale features like tagging or SSH access — the server-side implementation may be absent or broken for weeks after a client ships. Nebula's manual certificate workflow looks manageable until you have 30 nodes and need to rotate the CA. At that point you're writing your own tooling. ZeroTier's moon propagation delays are documented but the client-side behavior — where a client simply keeps routing through public roots even after a moon is configured — requires hands-on debugging to confirm it's actually working. None of these are dealbreakers, but all of them are costs that don't show up in the README.

Headscale: Tailscale Clients, Your Control Server

The thing that makes Headscale interesting isn't that it replaces Tailscale — it's that it reuses Tailscale. Your nodes run the official tailscale client binary, pointed at your own control server instead of Tailscale's. Headscale reimplements the coordination plane (key exchange, peer lists, ACL distribution) so the actual WireGuard tunnel setup stays identical to what you'd get from the vendor. That's a meaningful distinction: you're not shipping a new VPN stack to every node, you're just swapping where they phone home.

Getting it running is straightforward. A minimal Docker Compose setup that actually works:

services:
  headscale:
    image: headscale/headscale:0.23
    container_name: headscale
    volumes:
      - ./config:/etc/headscale
      - ./data:/var/lib/headscale
    ports:
      - "8080:8080"
      - "9090:9090"   # metrics
    command: serve
    restart: unless-stopped

The config.yaml has a lot of keys, but only a handful matter for a working deployment:

server_url: https://headscale.yourdomain.com   # what clients advertise to each other
listen_addr: 0.0.0.0:8080
metrics_listen_addr: 0.0.0.0:9090

db_type: sqlite
db_path: /var/lib/headscale/db.sqlite

dns_config:
  magic_dns: true
  base_domain: headnet.internal
  nameservers:
    - 1.1.1.1

SQLite is fine for anything under a few hundred nodes — don't reach for Postgres unless you're actually hitting write contention. Once the server is up, registering a node is one command after running tailscale up --login-server https://headscale.yourdomain.com on the client:

# grab the nodekey from the tailscale up output on the client, then on the headscale host:
headscale nodes register --user myuser --key nodekey:abc123...

MagicDNS works reliably — peers resolve each other by hostname without any additional DNS config on your end. What doesn't work cleanly is exit node support and anything in Tailscale's newer feature surface: Tailscale SSH, Funnel, and the App Connector are control-plane features Headscale hasn't caught up on. The Headscale team maintains a compatibility matrix in the repo, and it's worth checking it against the specific tailscale client version you're deploying before you commit. A mismatch between client version and Headscale version tends to manifest as silent registration failures or ACL enforcement behaving unexpectedly rather than clean error messages. The best fit here is a team that already knows Tailscale's UX and just wants the control plane on metal they own. If you need Tailscale SSH or Funnel, Headscale isn't a drop-in — those features don't exist at this layer yet.

Netbird: WireGuard Mesh With a Built-in Admin UI

The part that surprises most people: Netbird's self-hosted stack is actually three logical components — a signal server, a management server, and a STUN/TURN relay — but they collapse into a single netbird-management Docker image plus a separately deployed coturn instance. You're not stitching together five repos. The management image handles peer registration, ACL policy distribution, and the web dashboard. The signal server (bundled in the same image) is the lightweight coordination channel that bootstraps ICE negotiation between peers. Coturn is the fallback relay when direct P2P fails, and it's the one piece you manage separately.

The minimum viable docker-compose.yml block for the management server looks roughly like this:

services:
  netbird-management:
    image: netbirdio/management:latest
    restart: unless-stopped
    ports:
      - "443:443"
      - "33073:33073"   # signal gRPC
      - "10000:10000"   # management gRPC
    environment:
      - NETBIRD_MGMT_API_ENDPOINT=https://your-domain.example.com
      - NETBIRD_SIGNAL_URI=your-domain.example.com:10000
      # Leave the OIDC block out entirely if you want setup-key auth
      # NETBIRD_OIDC_CONFIGURATION_ENDPOINT=https://your-idp/.well-known/openid-configuration
      # NETBIRD_OIDC_CLIENT_ID=your-client-id
    volumes:
      - ./netbird-mgmt:/var/lib/netbird

Without the OIDC block, Netbird falls back to setup keys — you generate a key in the dashboard, paste it into the client on enrollment, done. For a homelab with five to fifteen nodes this is completely fine and removes an entire dependency (no Keycloak, no Authentik required at setup time). The SSO path is worth it if you're managing more nodes or want per-user audit trails, but it adds a real operational surface. The OIDC configuration endpoint must be reachable from the management container at startup, not just at login time — that burns people who configure it behind a private DNS that the container can't resolve.

The NAT traversal story is where Netbird diverges meaningfully from Tailscale. Tailscale uses DERP (a relay protocol they operate) as its fallback; you can self-host a DERP server but most people don't. Netbird uses ICE — the same Interactive Connectivity Establishment protocol that WebRTC uses — which has better peer-to-peer success rates specifically on symmetric NAT scenarios because it tries more candidate pairs aggressively. The tradeoff: when ICE fails and you fall through to TURN relay, you're running coturn, which means you're on the hook for its TLS certs, its UDP port exposure (3478 for STUN, 5349 for TURNS), and its resource cost under relay load. A coturn instance doing no active relay sits around 15 MB RSS. Under heavy relay traffic that number climbs proportionally to throughput, not node count.

On a small VM — a 1 vCPU / 2 GB instance is realistic — the idle footprint for the full stack is around 120 MB RSS total across management, signal, and coturn. That's comfortable on a 2 GB machine as long as you're not actively relaying significant traffic. The database backing management is SQLite by default, which is fine up to somewhere in the low hundreds of nodes; past that you'd want to look at the Postgres backend option. One gotcha that doesn't surface in the quickstart: the management container writes its SQLite file to /var/lib/netbird/, and if you forget to bind-mount that directory, a container restart wipes your entire peer registry. Mount it before you enroll your first node, not after.

Nebula: Flat Mesh Without Any Central Relay

Most mesh VPN tools still phone home to something — a coordination server, a relay, a SaaS dashboard. Nebula doesn't. The lighthouse is the closest thing to a central component, and all it does is help peers find each other's public IP and port. Once the handshake completes, the lighthouse is out of the picture entirely. Packets go peer-to-peer, always. That's a fundamentally different threat model and a fundamentally different ops story than Tailscale's DERP relay fallback.

The CA setup is genuinely two commands, and that's not marketing simplification — that's the actual workflow:

# Generate the CA once, store ca.key somewhere safe (not on any node)
nebula-cert ca -name 'homelab'

# Sign a cert for each node — this is the only auth mechanism
nebula-cert sign -name 'node1' -ip '192.168.100.1/24' -ca-crt ca.crt -ca-key ca.key

Every node gets a config.yaml, its own signed cert, and the CA's public cert. There's no token, no API key, no enrollment flow. If the cert is valid and signed by your CA, the node is in. If it's not, it's not. The config itself is about 60 lines for a typical node — lighthouse address, cert paths, firewall rules, and whether this node is a lighthouse. A minimal non-lighthouse config looks like this:

pki:
  ca: /etc/nebula/ca.crt
  cert: /etc/nebula/node1.crt
  key: /etc/nebula/node1.key

static_host_map:
  "192.168.100.254": ["your-vps-ip:4242"]  # lighthouse's nebula IP → public addr

lighthouse:
  am_lighthouse: false
  hosts:
    - "192.168.100.254"

listen:
  host: 0.0.0.0
  port: 0  # 0 = random ephemeral port, fine for non-lighthouses

firewall:
  outbound:
    - port: any
      proto: any
      host: any
  inbound:
    - port: any
      proto: icmp
      host: any

What you give up relative to Tailscale is real and worth stating plainly. Nebula has no automatic key rotation — when a cert expires (you set the duration at signing time with -duration 8760h for one year), you re-issue and redeploy manually. There's no GUI, no mobile app with a split-tunnel toggle, no ACL editor in a browser. The firewall rules live in each node's YAML. There's also no concept of exit nodes as a first-class feature. Nebula is a pure infrastructure primitive: it gives you an encrypted overlay network and nothing else. The operational surface is low, but the UX surface is zero.

The sweet spot is a fixed homelab topology — a NAS, a few servers, maybe a VPS, nodes that exist for months or years. Sign the certs with a two-year duration, deploy, and mostly forget it. The worst fit is a dynamic fleet where you're adding and removing devices frequently and want someone (or something) to handle re-enrollment. Every new node requires you to issue a cert, copy three files to it, and start the daemon. That's not hard, but it doesn't scale to a team or a setup where devices join and leave constantly. If your topology changes weekly, the lack of a management plane will grind on you fast.

ZeroTier Self-Hosted Controller and innernet: The Honorable Mentions

The "self-hosted ZeroTier" pitch sounds clean until you actually read the architecture. Running ztncui or posting to the /controller REST API on your own node gives you local network management — peer authorization, route assignment, all of it. But ZeroTier's planet/moon model still phones home to ZeroTier's root infrastructure for initial peer discovery unless you explicitly configure a private moon. That step is not prominent in most third-party setup guides. Without it, your "self-hosted" controller still depends on ZeroTier's uptime for new peers to find each other. Configuring a moon means generating a moon identity, hosting it on a reachable IP, and distributing the moon definition to every peer via zerotier-cli orbit. Doable, but it's a second project sitting on top of the first one.

# Generate a moon from an existing node's identity
zerotier-idtool initmoon identity.public > moon.json
# Edit moon.json to add your public IP under "stableEndpoints"
zerotier-idtool genmoon moon.json
# Output: 000000DEADBEEF.moon — copy to /var/lib/zerotier-one/moons.d/
# On every peer:
zerotier-cli orbit DEADBEEF DEADBEEF

innernet takes the opposite architectural stance — it's WireGuard all the way down, with a server component written in Rust that distributes peer configuration as TOML files. The peer invite flow is genuinely nicer than Nebula's CA ceremony. You run innernet add-peer <network> on the server, hand the resulting invite file to the new peer, and they run innernet install <invite.toml>. No intermediate CA steps, no manually signing certificates. Configuration ends up readable and diffable, which matters when you're debugging at midnight. The catch is the project's commit cadence. At the time of writing, the GitHub activity has stretched into multi-month quiet periods. That's not automatically fatal for stable software, but before you wire innernet into a production homelab, check the commit history yourself — specifically whether issues touching recent kernel versions are getting responses.

The honest framing for both: the technology underneath is sound. ZeroTier's virtual Ethernet model is mature and the controller API is well-documented. innernet's WireGuard-native design means the crypto and tunnel primitives aren't the risk surface. The risk is organizational. Headscale has active maintainers, a clear contribution pipeline, and enough community momentum that a single contributor going quiet doesn't stall the project. Netbird has a commercial entity behind it with obvious incentive to keep the open-source version functional. ZeroTier self-hosted and innernet carry higher bus-factor exposure — smaller contributor pools, and in innernet's case, no obvious commercial backing pushing maintenance forward. For a homelab where you can tolerate a slower patch cycle, that trade-off is acceptable. For anything you'd be paged about at 3am, weight that risk seriously before committing.

Picking the Right Tool for Your Setup

The fastest way to narrow this down: ask what you're actually replacing. If you're migrating an existing Tailscale deployment — you already have your ACLs defined, your peers are registered, your team knows the tailscale up workflow — Headscale is the obvious move. The client-side experience is identical. You swap the coordination server URL, re-auth your nodes, and you're done. No new mental model, no relearning. That migration friction is genuinely lower than any other option in this space, and that's not a small thing when you're managing more than a handful of nodes.

If you need a graphical management interface and real ACL tooling without hand-editing YAML for every peer relationship, Netbird is where you land. The UI is well-built, the policy model maps cleanly to how most people think about network segmentation, and you can self-host the full control plane. The honest cost is that you're now also operating coturn as infrastructure — that's your STUN/TURN relay, and it needs to stay up for peers behind symmetric NAT to stay connected. Plan for that. It's not hard to run, but it's another service to monitor, another port to keep open (3478/udp and 5349/tcp by default), and another thing that pages you at 2am if it silently dies.

For a static mesh where you know your topology won't change much and you want the smallest possible attack surface: Nebula. There's no control plane running anywhere. No API endpoint accepting connections. No coordination server to patch when a CVE drops. You generate certs with nebula-cert sign, distribute them out-of-band, point peers at your lighthouse IPs, and the network runs. The lighthouse is just a rendezvous point — it holds no state about your traffic. If your threat model includes "an attacker who can reach my control plane," Nebula removes that surface entirely. The tradeoff is that adding a new peer is a manual operation, every time.

The same evaluation logic extends cleanly to AI workloads and automation pipelines that need to run behind a mesh — whether that's an n8n instance talking to a local Ollama endpoint, or a semantic search service you don't want publicly exposed. For those setups, the local-vs-cloud tradeoff question doesn't stop at the model layer; it runs all the way down to the networking layer. The same reasoning that pushes you toward local models for data sensitivity reasons also pushes you toward self-hosted mesh networking. If you're working through that broader tradeoff, the AI Coding Tools in 2026: Cloud Copilots vs Local Models guide covers how these decisions stack across the toolchain — the criteria transfer directly.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.

5 Self-Hosted Monitoring Tools Worth Running on Your Own Hardware in 2026

우병수 — Wed, 15 Jul 2026 08:10:31 +0000

TL;DR: Most SaaS monitoring products are built around the assumption that you're running fleets of identical cloud VMs serving HTTP traffic. Plug in the agent, get CPU and memory graphs, call it done.

📖 Reading time: ~18 min

What's in this article

The Monitoring Problem No SaaS Tool Solves for You
Prometheus + Grafana: The Standard Stack That Earns Its Complexity
VictoriaMetrics: Prometheus-Compatible, Half the RAM
Netdata: Useful First, Configurable Later
Uptime Kuma: Exactly One Job, Done Well
Loki + Promtail: Log Aggregation That Doesn't Eat Your Disk
Picking the Right Tool for Your Actual Setup

The Monitoring Problem No SaaS Tool Solves for You

Most SaaS monitoring products are built around the assumption that you're running fleets of identical cloud VMs serving HTTP traffic. Plug in the agent, get CPU and memory graphs, call it done. That model breaks completely when your stack is an Ollama inference server hammering a 32GB VRAM workstation, an n8n instance running inside Docker with a dozen active workflows, and a PM2-managed Node engine firing on cron schedules. The per-host billing model punishes you twice: once for the cost, and again because the default dashboards surface nothing you actually care about.

The metrics that actually matter for this kind of setup don't exist in any default SaaS template. GPU VRAM headroom is the first thing I check — if nvidia-smi shows the model context is eating 28GB of 32GB and a second request hits, latency spikes aren't a mystery anymore, they're a predictable consequence. Container restart loops in Docker are a close second: a workflow executor that silently restarts every 40 minutes will look fine in an uptime check but corrupt half your pipeline runs. PM2 cron jobs fail quietly — no exit code surfaces unless you're explicitly scraping the process list. And reverse proxy 502 bursts that last under 30 seconds will disappear from any monitoring tool polling at 1-minute intervals.

The evaluation criteria for the five tools below are deliberately narrow. Install complexity matters because a monitoring setup that takes three days to configure is a monitoring setup you'll abandon. Resource footprint matters because the host you're monitoring is already doing real work — a 600MB resident-memory agent on the same box as an inference server is a bad trade. And the tool either surfaces ops-relevant data in under an hour of configuration, or it doesn't make the list. No partial credit for "you can build a dashboard that shows this if you write enough PromQL."

For context on what pipeline-level observability looks like — specifically how n8n workflow success and failure rates translate into something you can alert on — the framing in Workflow Automation in 2026: n8n, Zapier, and Self-Hosted Pipelines is worth reading alongside this. The monitoring layer and the automation layer solve different problems, but their failure modes overlap more than most ops writeups acknowledge. A cron pipeline that silently stops producing output looks identical to a cron pipeline that never ran — you need both execution telemetry and system metrics to tell them apart.

Prometheus + Grafana: The Standard Stack That Earns Its Complexity

The thing most people get wrong about this stack: Prometheus and Grafana don't integrate — they coexist. Prometheus scrapes exporters on a pull model and stores the resulting time-series locally. Grafana queries that store and draws pictures. There's no magic glue, no plugin that connects them invisibly. You wire them together at the datasource level, and you feel every seam when something breaks. That transparency is actually the argument for using it. When a dashboard shows nothing, you can bisect the problem: is the exporter up? Is Prometheus scraping it? Is the Grafana datasource pointed at the right port? Each layer is independently interrogatable.

Here's a minimum viable setup that actually runs. The provisioned datasource avoids the "click around in the UI" step that gets lost the moment someone rebuilds the container:

# docker-compose.yml
services:
  prometheus:
    image: prom/prometheus:v2.51.0
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.retention.time=90d'
      - '--storage.tsdb.wal-compression'  # cuts WAL disk use 30-40%
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana:10.4.2
    volumes:
      - grafana_data:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=changeme
    ports:
      - "3000:3000"
    depends_on:
      - prometheus

volumes:
  prometheus_data:
  grafana_data:

# prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets: ['node_exporter:9100']

  - job_name: 'cadvisor'
    static_configs:
      - targets: ['cadvisor:8080']

# grafana/provisioning/datasources/prometheus.yaml
apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    url: http://prometheus:9090  # container name, not localhost
    isDefault: true
    editable: false

The retention math is the part nobody thinks about until they're watching df -h scroll upward. At 15-second scrape intervals, a single exporter like node_exporter generates roughly 1–2 MB of TSDB data per day. Forty exporters — a realistic number once you're scraping per-container metrics with cAdvisor across several hosts — means 40–80 MB/day, which compounds quickly against a 90-day retention window. The --storage.tsdb.wal-compression flag pairs with --storage.tsdb.retention.time=90d and meaningfully reduces the write-ahead log on disk. If you're pushing past that, the Prometheus docs point toward Thanos or VictoriaMetrics for remote storage, but that's a different problem entirely.

Pick this stack when the requirement is a permanent, queryable, auditable record of your infrastructure — something you can run ad-hoc PromQL against at 2am to figure out what happened three weeks ago. The query language has a learning curve that's real but finite. Once you can write rate(http_requests_total[5m]) and understand why the range vector matters, the model clicks. If the requirement is instead "get a dashboard running before the next standup," this stack will frustrate you. The provisioning alone assumes you understand what a datasource is and why it needs to be inside the Grafana container's filesystem on startup. That's not a criticism — it's a signal about which kind of operator this tool is designed for.

VictoriaMetrics: Prometheus-Compatible, Half the RAM

Most people discover VictoriaMetrics the wrong way — they see "Prometheus-compatible" and assume it's just a faster remote storage backend. The more useful mental model: the single-node binary is the storage and the scrape target rolled into one process. Drop it in place of Prometheus, point your existing prometheus.yml at it, and your Grafana dashboards keep working without a single panel edit. MetricsQL is a superset of PromQL, so queries that work in Prometheus work here too — the extension functions are additive, not breaking.

The RAM story is real but requires nuance. On a scrape config hitting 50 targets at a 15-second interval, VictoriaMetrics single-node typically sits under 200 MB RSS. Prometheus doing the same work tends to land in the 400–600 MB range, sometimes higher depending on how aggressive your recording rules are. The official docs claim up to 7x less RAM, which is technically achievable under ideal conditions — low cardinality, compact label sets. Where the gap closes is high-cardinality data: if you're scraping something that emits per-request or per-user label dimensions, both tools suffer proportionally and the ratio shrinks toward 2–3x. Still meaningful on a constrained machine, but not magic.

The configuration model is deliberately minimal, which is either refreshing or annoying depending on your background. Retention is a process flag, not a YAML stanza:

# docker-compose fragment
services:
  victoriametrics:
    image: victoriametrics/victoria-metrics:v1.101.0
    command:
      - "--storageDataPath=/victoria-metrics-data"
      - "--retentionPeriod=12"   # months, not days — easy to misread
      - "--httpListenAddr=:8428"
    ports:
      - "8428:8428"
    volumes:
      - vm-data:/victoria-metrics-data

If you want the collection layer decoupled from storage — useful when you're scraping dozens of targets and want independent scaling — vmagent is the sidecar that handles scraping and remote_write forwarding. It accepts the same scrape_configs format Prometheus does. The split also means you can restart the storage node for maintenance without dropping scrape coverage, which Prometheus's monolithic design doesn't give you cleanly.

Pick VictoriaMetrics over Prometheus when at least one of these is true: you're running on hardware where RAM has real consequences (a mini-PC lab node, a shared VPS where you're paying per GB), you have existing Grafana dashboards you're not willing to rewrite, or you need retention beyond 30 days without provisioning proportionally large disk. Prometheus's default storage compresses reasonably well but still scales linearly with retention window. VictoriaMetrics's storage engine compresses more aggressively — the on-disk footprint for the same dataset is noticeably smaller, which matters when you're trying to keep 6–12 months of metrics on a 50 GB volume.

Netdata: Useful First, Configurable Later

The install story here is genuinely unusual. Most monitoring tools require you to configure a scrape target, set up a service, restart things three times, and then realize you forgot to open a firewall port. Netdata's kickstart script does none of that:

# drops a working dashboard at http://your-host:19999 — no follow-up config required
wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh && sh /tmp/netdata-kickstart.sh

Under two minutes to a live dashboard. It auto-discovers Docker containers by reading the socket, picks up systemd unit states, and surfaces metrics for Nginx, Postgres, Redis, and most common services without a single config file edit. If you've ever bootstrapped Prometheus from scratch — writing a prometheus.yml, finding the right exporters, wiring up Grafana datasources, building dashboards — the contrast is jarring. That zero-to-visible speed matters most during incidents when you inherit a broken machine and need situational awareness immediately.

Where Netdata genuinely outpaces a default Prometheus stack is metric resolution and pre-built visualizations. Prometheus node_exporter collects CPU steal time, disk latency percentiles, and TCP retransmit rates — but at a 15-second scrape interval by default, and without shipping any dashboards. You get the data; you build the panels. Netdata ships those visualizations pre-wired at one-second resolution. CPU steal spikes that last four seconds show up clearly. A Prometheus setup scraping every 15 seconds can miss those entirely, or flatten them into a near-invisible bump. For the specific problem of catching short-duration resource contention on shared infrastructure, that resolution difference is decisive.

The retention story is where you have to make a deliberate decision before committing. Netdata's native dbengine is designed for short-term forensics — excellent for the last few hours, workable for a few days, not designed for 90-day trending or capacity planning queries. Your two options for extending retention both have costs. First, configure a prometheus remote_write target and push metrics into your existing Prometheus/VictoriaMetrics stack:

# /etc/netdata/exporting.conf
[prometheus_remote_write:my_victoria]
    enabled = yes
    destination = http://victoriametrics:8428/api/v1/write
    remote write URL path = /api/v1/write
    # send everything; filter later at query time
    send charts matching = *

Second option is Netdata Cloud, which streams your metrics to their hosted infrastructure. For operators running self-hosted specifically because they want data on their own hardware — air-gapped environments, compliance constraints, or just preference — that option is a non-starter. The remote_write path keeps you in control, but now you're running two systems and the "zero config" advantage is diluted. Pick Netdata when you need immediate visibility with no setup friction, when you're monitoring a secondary node where building a full Prometheus stack isn't worth the overhead, or when per-second granularity is the actual requirement rather than long-term trend analysis.

Uptime Kuma: Exactly One Job, Done Well

The Prometheus stack will tell you your p99 latency drifted 40ms — but if Nginx is spitting 502s at every visitor while your internal health endpoint still responds on port 8080, you might not find out until someone complains. Uptime Kuma solves exactly that gap: external black-box polling that tests what a real client actually sees. It doesn't try to be a metrics platform. That restraint is the whole point.

Check types cover the common cases without bloat: HTTP/HTTPS with configurable expected status codes and keyword matching, TCP port reachability, DNS record resolution, and Docker container up/down status. If you need distributed tracing or cardinality-rich time series, look elsewhere. If you need to know whether your VPN endpoint is accepting connections from the outside, Uptime Kuma answers that in under thirty seconds of setup.

The deploy is one command:

docker run -d \
  --restart=always \
  -p 3001:3001 \
  -v uptime-kuma:/app/data \
  --name uptime-kuma \
  louislam/uptime-kuma:1

Everything — monitors, alert configs, notification channels, status history — lives in a single SQLite file inside that named volume. Backup strategy is literally cp uptime-kuma.db uptime-kuma.db.bak. No external database to manage, no schema migrations to babysit across upgrades. The tradeoff is that SQLite doesn't scale to hundreds of monitors with sub-second polling intervals, but for a self-hosted stack watching thirty to fifty endpoints that ceiling is not a real concern.

The alerting integrations are where it earns its slot alongside a full metrics stack. Telegram, Discord, Slack, and generic webhooks are all first-class. The webhook path drops cleanly into n8n — POST the payload to an n8n webhook trigger, route on monitor name or status, and you have conditional logic, escalation delays, or incident log writes without touching Uptime Kuma's internals at all. Configuration for any of these takes five minutes, not fifty.

One gotcha that doesn't surface until you try the Docker container monitor: it requires mounting the Docker socket into the Uptime Kuma container.

docker run -d \
  --restart=always \
  -p 3001:3001 \
  -v uptime-kuma:/app/data \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --name uptime-kuma \
  louislam/uptime-kuma:1

Mounting /var/run/docker.sock gives the container effective root on the host — any process inside it can spawn, stop, or inspect any container on the machine. On a firewalled home-lab box where you control all ingress, that risk profile is manageable. On a VPS with a public IP and Uptime Kuma's web UI exposed directly (even behind basic auth), evaluate carefully. The container status monitor is convenient; it isn't worth a compromised host to get it.

Loki + Promtail: Log Aggregation That Doesn't Eat Your Disk

The real reason to add Loki to a self-hosted stack isn't log storage — it's log recall. docker logs --tail 100 n8n gives you the last hundred lines right now, but if your n8n workflow executor threw a 500 at 2 AM and you're looking at it at 9 AM, those lines are gone. Either you have log rotation eating them, or the buffer scrolled past them hours ago. Tracking that down with grep across rotated /var/lib/docker/containers/**/*-json.log files is painful enough once that you'll set up Loki immediately afterward. The architectural reason Loki doesn't balloon your disk is that it indexes only labels — not the full log text. The log lines themselves get compressed and stored as chunks; the index is tiny. Full-text search comes at query time via regex on the stored chunks, not via an inverted index the way Elasticsearch works. That trade-off means slower arbitrary searches but dramatically lower storage overhead for typical DevOps log volumes.

Getting Promtail to auto-discover Docker containers takes about 20 lines of config. The key is docker_sd_configs, which reads the Docker socket and emits a scrape target per running container. Relabeling pulls container_name and com.docker.compose.service out of the discovered metadata and turns them into Loki labels:

server:
  http_listen_port: 9080

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: docker
    docker_sd_configs:
      - host: unix:///var/run/docker.sock
        refresh_interval: 15s
    relabel_configs:
      # keep the bare container name (strips leading slash Docker adds)
      - source_labels: [__meta_docker_container_name]
        regex: "/(.*)"
        target_label: container_name
      # pull the Compose service label if it exists
      - source_labels: [__meta_docker_container_label_com_docker_compose_service]
        target_label: compose_service
      # standard log path Promtail needs to tail
      - source_labels: [__meta_docker_container_log_stream]
        target_label: stream

Mount /var/run/docker.sock read-only into the Promtail container and this works without touching individual container configs. After it's running, {container_name="n8n"} in Grafana's Explore view returns every log line that n8n has emitted since Promtail started. You can narrow it to {container_name="n8n"} |= "500" or pipe through a regex for specific workflow IDs. The label cardinality stays low because container names and Compose service names are a short, stable set — exactly what Loki's index is designed for.

The part the official getting-started docs consistently underplay: Loki will accumulate index chunks indefinitely unless you explicitly configure the compactor to run retention. The boltdb-shipper compactor defaults to compaction only, not deletion. You need two separate config blocks to actually enforce retention:

compactor:
  working_directory: /loki/compactor
  shared_store: filesystem
  compaction_interval: 10m
  retention_enabled: true        # this flag is the non-obvious one
  retention_delete_delay: 2h
  retention_delete_worker_count: 150

limits_config:
  retention_period: 744h         # 31 days; set per-tenant or globally here

Without retention_enabled: true in the compactor block, the retention_period value in limits_config does nothing. The chunks and index keep growing. Disk fills up on a timeline that depends on your log volume, but on a moderately busy Docker host running a dozen containers, expect several gigabytes per week before compression. With retention enabled and 31 days configured, disk usage plateaus and stays there. Check that it's actually deleting by watching loki_compactor_deleted_chunks_total in Prometheus — if that counter never moves, the compactor config isn't being picked up.

The right time to add Loki is exactly when you already have Prometheus and Grafana running. You add Loki as a second datasource in Grafana's datasource settings — same UI, same dashboards panel editor, just a different query language (LogQL instead of PromQL). The marginal operational cost is one more container plus Promtail, and the compactor config above. What you get back is the ability to correlate a spike in an HTTP 500 metric on a Prometheus graph with the exact log lines that caused it, in the same Grafana window, without switching tools. That's the payoff: not fancy log analytics, just not being blind at 2 AM.

Picking the Right Tool for Your Actual Setup

The honest answer to "which tool should I use" is that the decision almost always comes down to two constraints that people don't state up front: how much idle RAM they can spare on the host, and whether they already have a query/visualization layer or need one bundled in. Everything else — cardinality limits, retention policies, integrations — is secondary until you've cleared those two gates.

Here's how the five tools stack up across the dimensions that actually matter for a self-hosted setup:

Tool

Primary Use Case

Storage Model

Approx. Idle RAM

Retention Flexibility

Biggest Operational Gotcha

Prometheus

Metrics scraping + alerting

Local TSDB (pull-based)

~250–400 MB

Fixed retention flag; no tiered storage natively

High-cardinality label sets will eat RAM fast; no horizontal scale without remote write

VictoriaMetrics

Long-term metrics storage

Custom columnar TSDB

~50–120 MB (single-node)

Per-metric TTL, downsampling, configurable at ingest

MetricsQL is close to PromQL but not identical — existing dashboards need audit

Grafana

Unified query + visualization

No native metrics store

~150–250 MB

Depends entirely on backend datasource

Dashboard state stored in SQLite by default — back it up before any container restart

Netdata

Real-time system telemetry

In-memory ring buffer + optional DB engine

~150–300 MB (scales with metrics count)

Short by default; DB engine required for >1 day

Cloud-connected by default; disable claim or it phones home on first run

Uptime Kuma

Endpoint + uptime checks

SQLite

~60–100 MB

Configurable per-monitor history trim

No native metric federation; alerting is push-only with no query interface

Loki

Log aggregation + querying

Object store or local filesystem chunks

~100–200 MB (single-binary)

Retention via compactor; per-stream rules possible

LogQL regex on high-volume streams is slow without good label strategy upfront

The decision path by constraint is fairly mechanical once you're honest about your situation. RAM-limited host running under 4 GB free — go VictoriaMetrics over Prometheus, full stop. The idle footprint difference is real and compounds when you're also running Grafana, a Node process, and Docker overhead on the same box. Need immediate visibility into a new host with zero config written — Netdata is the only one that installs and shows you useful data in under five minutes; everything else requires you to define scrape targets or shipping configs first. Need uptime alerting wired into an existing webhook or n8n flow — Uptime Kuma's webhook output is dead simple and requires no intermediate exporter layer. Need to correlate "the API started failing" with "what was in the logs at that exact timestamp" — Loki alongside whichever metrics backend you chose, queried together inside Grafana. That combination is the only way to close that loop without grep-ing through raw log files by hand.

For a single-workstation home lab running Ollama, Docker services, and a Node automation engine — which is exactly my setup — the stack I'd recommend and actually run is: Uptime Kuma handling external endpoint checks and pushing alerts to n8n webhooks, VictoriaMetrics as the metrics backend because it handles Prometheus remote_write and stays light, Grafana as the single query interface pointed at both VictoriaMetrics and Loki, and Loki added the moment you've had one debugging session where you were correlating container restarts to application errors without correlated timestamps. That last condition sounds vague until it happens, and then Loki goes in immediately. The docker-compose fragment to wire VictoriaMetrics as Prometheus remote_write target looks like this:

# prometheus.yml scrape config with remote_write to VictoriaMetrics
# VictoriaMetrics accepts the /api/v1/write endpoint natively
remote_write:
  - url: http://victoriametrics:8428/api/v1/write
    queue_config:
      max_samples_per_send: 10000
      # tune this down on low-traffic hosts to reduce write amplification
      batch_send_deadline: 5s

One boundary worth being explicit about: none of these tools — not Netdata, not VictoriaMetrics, not any combination of them — will give you per-request Ollama inference latency or VRAM fragmentation state between model loads out of the box. Ollama exposes /api/tags for loaded model state and you can scrape process-level GPU stats via nvidia-smi --query-gpu=memory.used,memory.free --format=csv,noheader on a cron or as a custom exporter, but stitching that into meaningful per-request latency tracking requires either a sidecar exporter or middleware instrumentation at the API layer. That's a different problem and a separate article — don't let the absence of it here suggest any of these five tools are incomplete. They're solving infrastructure visibility, not ML observability.

Originally published on techdigestor.com. Follow for more developer-focused tooling reviews and productivity guides.