Artem Meleshkin

Posted on Aug 28 Originally published at pingvera.com

Monitoring as Code for Web Agencies — When and How

#webdev #monitoring #wordpress #sre

Monitoring as Code for Web Agencies: When and How

Monitoring as code means storing the intended monitoring configuration in a versioned, reviewable format and applying it through an API, CLI, provider, or controlled automation. It is useful when an agency has repeatable service classes, many similar client sites, frequent onboarding, and configuration drift that manual review can no longer control.

It is not automatically better for a ten-site portfolio with unstable standards. Automating inconsistent monitoring creates inconsistent monitoring faster.

At a glance

Use monitoring as code when the agency needs:

repeatable monitor bundles;
peer-reviewed changes;
history and rollback;
bulk policy updates;
drift and exception visibility;
automated onboarding/offboarding;
consistent routing and metadata.

Keep secrets outside the repository, test generated configuration, use canary rollout, and prevent automation from deleting or silencing production monitors without safeguards.

What belongs in code

service class and monitor template version;
URL/path, method, expected status/content, frequency, timeout;
safe synthetic request definition;
TLS/domain thresholds;
regions and confirmation logic;
severity, owner, client/site IDs, and alert route references;
maintenance-window reference;
exception ID and expiry;
help-desk/status-page mapping;
disabled state with reason and expiry.

Do not commit:

passwords, API tokens, private keys, mailbox credentials;
production customer data;
unredacted webhook samples containing sensitive information;
secrets hidden in URLs or request bodies.

A vendor-neutral internal schema

The following YAML is an agency-owned desired-state format, not a claim about Pingvera or another vendor's API.

schema_version: 1
site_id: WEB-0042
service_class: A
owners:
service: client-success
technical: maintenance-platform
alert_route: oncall-critical

checks:

id: canonical-home
type: http_content
url: https://shop.example.com/
interval_seconds: 60
timeout_seconds: 10
expected_status: 200
expected_text: "Shop Example"
regions: [us-east, eu-west]
confirm_after: 2
severity: P2
id: lead-delivery
type: form_delivery
form_url: https://shop.example.com/contact
interval_seconds: 300
payload_secret_ref: vault://clients/WEB-0042/form-test
destination_secret_ref: vault://clients/WEB-0042/test-mailbox
delivery_deadline_seconds: 180
severity: P1

expiry:
tls_days: [30, 14, 7]
domain_days: [60, 30, 14]

exceptions:

id: EX-019 check: lead-delivery reason: "Client migration window" expires_at: 2026-09-01T12:00:00Z approved_by: client-service-owner

Repository structure
monitoring/
├── schemas/
│ └── site-monitoring-v1.json
├── templates/
│ ├── class-a-store.yaml
│ ├── class-b-lead-site.yaml
│ └── class-c-content.yaml
├── sites/
│ ├── WEB-0042.yaml
│ └── WEB-0043.yaml
├── policies/
│ ├── alert-routes.yaml
│ └── maintenance-windows.yaml
├── tests/
│ ├── required-checks.yaml
│ └── fixtures/
└── README.md

Separate reusable templates from site parameters and explicit exceptions.

Change workflow

inventory change
→ generate/edit desired state
→ schema and policy validation
→ peer review
→ dry run / diff
→ canary apply
→ external validation
→ staged rollout
→ evidence and inventory update

Require human approval for high-blast-radius changes: deletion, mass disablement, alert-route changes, reduced frequency, weakened success condition, or changed P1 suppression.

Tests to run before apply

Schema tests

required fields and valid types;
valid URLs, time zones, UTC timestamps, and severity values;
unique site and check IDs;
exception expiry present;
no plaintext-secret patterns.

Policy tests

active Class A/B site has a critical journey;
P1/P2 route has a fallback;
suppression cannot exceed policy maximum;
domain/TLS thresholds meet the class standard;
form/checkout test has synthetic-data and cleanup metadata;
archived site cannot receive normal on-call routing.

Behaviour tests

monitor endpoint is reachable from test context;
expected marker is specific enough;
synthetic action cannot trigger a real charge/shipment;
test event reaches the correct route;
apply is idempotent;
deleted configuration requires explicit approval.

Drift and source of truth

Decide whether:

code is authoritative and UI changes are prohibited/reverted;
UI changes are imported into code through review;
selected fields are managed manually and ignored by automation.

Without a declared model, engineers will fight the tool: code overwrites emergency changes, while the UI creates invisible drift.

Emergency UI changes should create an incident/change record and a follow-up pull request.

Safe rollout

begin with read-only export and diff;
onboard a small representative portfolio;
apply one template change to canaries;
verify monitoring and alert delivery;
roll out in batches;
stop on unexpected deletion, route change, or alert spike;
retain a previous desired state and provider-side evidence;
review exceptions and drift after rollout.

Do not deploy a new monitor template to 100 sites at the same moment as a CMS bulk update.

API and automation reliability

respect rate limits and pagination;
use idempotent operations where supported;
persist provider IDs mapped to internal site/check IDs;
retry temporary failures with bounded backoff;
distinguish “request accepted” from “monitor active”;
reconcile desired and observed state;
log changes without logging secrets;
monitor the automation itself;
require explicit handling for partial bulk failure.

When not to use monitoring as code

portfolio is small and changes are rare;
service classes and required checks are not defined;
provider lacks a stable supported API/export;
nobody owns the repository and failed applies;
manual UI configuration is currently faster and fully reviewable;
automation risk exceeds current configuration drift.

A structured inventory and manual template checklist may be the correct intermediate stage.

Example: expired exceptions

An agency disables checkout monitoring during a client migration and forgets to restore it. In code, the exception requires an expires_at value. Policy validation fails if expiry is absent, and the reconcile job surfaces the expired exception even if the provider leaves the check paused.

The benefit is not YAML. It is an enforceable lifecycle for deviations.

Common mistakes

repository filled with copied full configurations instead of templates;
secrets committed in headers or URLs;
no dry run or destructive-change approval;
code and UI both treated as authoritative;
assuming API success means check success;
bulk rollout without canaries;
no owner for failed or partial applies;
automating before defining service classes;
measuring lines of configuration instead of coverage and alert quality.

FAQ

Is Terraform required?

No. A supported provider, CLI, API client, GitOps workflow, or internal reconciler can implement the pattern. Choose the least complex reliable option.

How many sites justify monitoring as code?

There is no fixed number. Repetition, change rate, compliance needs, and drift cost matter more than site count.

Should clients see the configuration repository?

Provide appropriate transparency without exposing other clients, internal routes, or sensitive metadata. Client-specific exports may be safer.

How do we handle an urgent manual change?

Allow a documented emergency path, then reconcile it into the desired state promptly.

Sources and further reading

Reviewed: 8 August 2026.

Next: Send Pingvera events to a help desk and manage multiple client websites.

Before building automation around Pingvera, confirm the current supported API, CLI, export, and webhook capabilities. Do not design against undocumented fields.

Originally published at pingvera.com.

DEV Community