DEV Community: Diego Gil Serrano

What NIS2 Article 21 actually asks you to log, and how to do it with a self-hosted SIEM

Diego Gil Serrano — Sun, 25 Oct 2026 05:51:39 +0000

If you run infrastructure for a company that falls under NIS2 (or sells to one), you will eventually be asked: "Are we logging what NIS2 requires?" The Directive itself does not contain the word SIEM. But read Articles 21 and 23 together with the Commission's Implementing Regulation (EU) 2024/2690 and the logging requirements become quite specific.

This post maps those texts to concrete log sources and events, and shows how to cover them with tools you host yourself.

This is a practical summary, not legal advice. Always check the official text and your national transposition.

Where logging appears in the Directive

Article 21(1) requires essential and important entities to take "appropriate and proportionate technical, operational and organisational measures" to manage risks to their network and information systems. Article 21(2) lists the minimum set. Three items depend directly on logs:

(b) incident handling. You cannot detect, analyse or contain an incident without a record of what happened.
(f) policies and procedures to assess the effectiveness of cybersecurity risk-management measures. Logs are how you prove a control works, not just that it exists.
(j) the use of multi-factor authentication or continuous authentication solutions. Authentication events are the evidence that MFA is actually enforced.

Article 23 adds the clock: an early warning within 24 hours of becoming aware of a significant incident, an incident notification within 72 hours, and a final report within one month of the notification. Producing a reliable timeline in 24 hours is very hard if logs are scattered over dozens of machines with drifting clocks.

Where it gets concrete: Implementing Regulation 2024/2690

The Implementing Regulation details the Article 21 measures for a specific set of digital providers: DNS service providers and TLD registries, cloud computing, data centres, content delivery networks, managed service providers, managed security service providers, online marketplaces, search engines, social networks and trust service providers. For everyone else it is not binding, but it is the most precise public description of what "incident handling" means in practice, and auditors use it as a reference.

Point 3.2 of its annex ("monitoring and logging") asks entities to keep logs that include, where appropriate:

relevant outbound and inbound network traffic;
creation, modification or deletion of users and extension of permissions;
access to systems and applications;
authentication-related events;
all privileged access and activities performed by administrative accounts;
access or changes to critical configuration and backup files;
event logs and logs from security tools such as antivirus, IDS or firewalls;
use of system resources and their performance;
physical access to facilities;
access to and use of network equipment and devices;
activation, stopping and pausing of the various logs;
environmental events.

It also asks for regular review of logs, automated alarms where appropriate, a defined retention period, protection against unauthorised access or modification, synchronised clocks, and monitoring that is reliable and does not depend on the systems it watches.

Mapping requirements to real sources

Requirement	Typical source	What to collect
Network traffic	Firewall, VPN concentrator, reverse proxy / WAF	Allowed and denied connections, VPN sessions, admin logins to the firewall
User lifecycle and permissions	Active Directory / LDAP, Linux `/etc/passwd` changes	Windows events 4720 (user created), 4726 (user deleted), 4728 / 4732 / 4756 (member added to a security group)
Authentication	Domain controllers, Linux `auth.log` / `secure`, SSH, web apps	4624 (logon), 4625 (failed logon), 4740 (account locked out), `sshd` accepted/failed
Privileged activity	`sudo`, Windows admin groups, hypervisor (ESXi) logs	`sudo` commands, admin logins to vCenter/ESXi hosts
Critical config and backups	Backup server, config management, storage arrays	Job results, deleted snapshots, config changes
Security tools	Antivirus/EDR, IDS, firewall	Detections and policy changes
Logging stopped	Windows, rsyslog, agents	Event 1102 (audit log cleared); "source silent for N minutes" alerts

The last row is the one most setups miss. Requirement 11 is easiest to meet with an alert that fires when a source stops sending.

A minimal self-hosted pipeline

You do not need a large platform to start. The pieces:

1. Time. Same NTP source everywhere, and check drift:

chronyc tracking | grep -E 'System time|Leap status'

2. Forwarding from Linux. With rsyslog, forward everything over TCP to your collector:

# /etc/rsyslog.d/90-forward.conf
*.* @@logs.example.internal:514

Use TLS (omfwd with StreamDriver="gtls") if logs cross untrusted networks.

3. Windows. Enable the advanced audit policy subcategories for logon, account management and security group management, and ship the Security log with an agent or Windows Event Forwarding.

4. Network devices and hypervisors. Firewalls, switches and ESXi hosts all speak syslog; point them at the collector.

5. Detection rules you can explain to an auditor. Start with a handful:

Brute force: more than N failed logons for one account from one source in M minutes.
Password spraying: one source failing against many different accounts in a short window.
Lockouts: any 4740 on a privileged account.
Out-of-hours admin logins.
Changes to admin groups (4728/4732/4756 on Domain Admins and equivalents).
Source silent: no events from a host for longer than its normal interval.

6. Review and tickets. Every alert should land somewhere with an owner, a status and a history. "We review logs" is a claim; a list of reviewed alerts with dates and names is evidence.

7. Retention and integrity. Neither the Directive nor the Regulation fixes a number of days. They ask for a defined period, justified by your risk assessment and other legal obligations. Document it, enforce it, and restrict who can read and delete logs.

Why self-hosting matters here

Logs contain usernames, IP addresses, hostnames and sometimes personal data. Shipping all of that to a third-party cloud brings its own contractual, data-protection and supply-chain questions (supply chain security is itself Article 21(2)(d)). Many public-sector and regulated organisations prefer, or are required, to keep the data on their own infrastructure. A self-hosted collector avoids that discussion entirely, at the cost of operating it yourself.

Checklist

[ ] List of log sources in scope, and an alert when any of them goes silent
[ ] Authentication, user lifecycle and admin-group events collected from every domain controller
[ ] Firewall and VPN logs centralised
[ ] Common NTP source and drift check
[ ] Detection rules for brute force, spraying, lockouts, out-of-hours admin activity
[ ] Alerts tracked as tickets with owner and history
[ ] Documented retention period and restricted access to logs
[ ] A tested way to produce an incident timeline within 24 hours

Disclosure: I work on SIEM Compliance, a self-hosted SIEM from NUTRADELIS S.L. (Spain). Its Community edition is free with no limit on servers, agents or log volume, collects logs from firewalls, Windows and Linux servers, ESXi, switches and Cloudflare, and ships with rules like the ones above. Paid plans add an audit module with NIS2 and ENS evidence packs. The original guide covers the same topic with a Spain-specific section.

Sources

Directive (EU) 2022/2555 (NIS2), Articles 21 and 23, EUR-Lex: https://eur-lex.europa.eu/eli/dir/2022/2555/oj
Commission Implementing Regulation (EU) 2024/2690, Annex, point 3.2, EUR-Lex: https://eur-lex.europa.eu/eli/reg_impl/2024/2690/oj
Article 21 text (secondary reproduction used for cross-checking): https://luxgap.com/lois/nis2/art-21/?lang=en
Annex point 3.2.3 list of logged events (secondary reproduction used for cross-checking): https://www.cyberday.ai/requirement/nis2-guide-3-2-3-log-management-and-review
Microsoft, Windows security audit events reference: https://learn.microsoft.com/en-us/windows/security/threat-protection/auditing/security-auditing-overview
SIEM Compliance, original guide: https://www.siemcompliance.com/recursos/nis2-registro-monitorizacion-eventos/?lang=en

Why Patroni failover takes 26–49 seconds with default settings (and how to make it faster)

Diego Gil Serrano — Sun, 25 Oct 2026 05:41:37 +0000

You build a Patroni + etcd + HAProxy cluster, kill the primary in a test, and your write loop stops for half a minute or more. Nothing is broken. That is what the default settings are designed to do. This post explains where those seconds go and what you can change.

The three settings that matter

All three live in Patroni's dynamic configuration (stored in the DCS, changed with patronictl edit-config). From the official documentation:

Setting	What it is	Default	Minimum
`loop_wait`	"the number of seconds the loop will sleep" between HA cycles	10	1
`ttl`	the TTL of the leader lock; "think of it as the length of time before initiation of the automatic failover process"	30	20
`retry_timeout`	"timeout for DCS and PostgreSQL operation retries"; "DCS or network issues shorter than this will not cause Patroni to demote the leader"	10	3

The documentation also sets a hard rule when you change them:

loop_wait + 2 * retry_timeout <= ttl

With defaults: 10 + 2 × 10 = 30 ≤ 30. The defaults sit exactly on the limit.

What happens when the primary dies

While healthy, the primary renews the leader key in etcd on every HA loop, roughly every loop_wait (10 s).
The primary crashes (power loss, kernel panic, hard network isolation). Nobody renews the key.
The key lives until ttl (30 s) after its last renewal. Since renewals happen every ~10 s, the failure can land anywhere in that cycle: the key expires between ~20 s and ~30 s after the failure.
A replica notices the key is gone on its next HA cycle (between 0 and loop_wait seconds later), checks it is healthy enough (for example within maximum_lag_on_failover), takes the key and promotes PostgreSQL.
HAProxy, which polls each node's GET /primary (HTTP 200 only on the node "running as the primary with leader lock"), needs one or more successful checks on the new primary before it routes writes there. With checks every 2 s and a rise greater than 1, that adds a few more seconds.

Add up the pieces: 20–30 s for the key to expire, 0–10 s for a replica to notice, a second or two to promote, and a few seconds of health checks. You get the 26–49 s range you see with plain Patroni + etcd defaults and HAProxy checking every 2 s. The exact figure depends on where in the renewal cycle the failure happens, on how fast promotion completes, and on your health check settings.

Two important exceptions

Clean shutdown is fast. If you stop Patroni normally (systemctl stop patroni, a planned reboot) it releases the leader key on the way out, so a replica can take over without waiting for the TTL. That is why "I rebooted the primary and failover took 5 seconds" and "I pulled the power cable and it took 40" are both true.
A frozen primary is the slow case. A node that hangs (stuck I/O, process stopped) may keep its key until it expires, and it is still in HAProxy's backend until checks fail. This is the scenario to test, because it is the one that hurts most.

How to make it faster

1. Shrink `ttl`, `loop_wait` and `retry_timeout` together

For example:

# patronictl -c /etc/patroni/patroni.yml edit-config
loop_wait: 5
retry_timeout: 5
ttl: 20

5 + 2 × 5 = 15 ≤ 20, so the rule holds, and the worst-case key expiry drops from ~30 s to ~20 s after the last renewal. ttl cannot go below 20.

The trade-off is real:

A shorter retry_timeout means a shorter etcd or network hiccup is enough to demote a healthy primary. If your etcd runs on slow disks or across a flaky link, you will trade slow failovers for unnecessary ones.
A shorter loop_wait means more frequent DCS writes and more work per node.

Measure your etcd latency (etcdctl check perf, fsync metrics) before going aggressive.

2. Tune the HAProxy health checks

The usual Patroni HAProxy backend looks like this:

backend primary
    option httpchk GET /primary
    http-check expect status 200
    default-server inter 2s fall 3 rise 2 on-marked-down shutdown-sessions
    server db1 10.0.0.11:5432 check port 8008
    server db2 10.0.0.12:5432 check port 8008

inter × rise is the minimum time before the new primary receives traffic, and inter × fall the time before the old one is taken out. on-marked-down shutdown-sessions closes existing connections to a node marked down, so clients do not keep writing into a dead socket. Lowering inter to 1 s is cheap; it only adds HTTP requests to the Patroni REST API.

3. Make clients reconnect quickly

Database failover time is not application recovery time. Check your driver's connect timeout, TCP keepalives and retry logic. A pool that waits 2 minutes on a half-open TCP connection will hide every second you saved in Patroni.

4. Handle the frozen primary explicitly

Lower TTLs help, but a hung node is a detection problem, not just a timer problem. Options include Patroni's watchdog support (so a node that stops running its HA loop is reset) and external fencing that removes the old primary from every load balancer as soon as a failover is decided.

5. Measure, do not assume

Run a write loop through the load balancer and look at the gap:

while true; do
  psql -h VIRTUAL_IP -p 5000 -U app -d app -qAt \
    -c "INSERT INTO ha_test DEFAULT VALUES RETURNING now()" 2>&1
  sleep 0.5
done

Then try a switchover, a clean stop, a kill -9, and a SIGSTOP of Patroni and PostgreSQL on the primary (test environments only). Each one exercises a different path.

Summary

With defaults, the leader key expires 20–30 s after a hard failure; add detection, promotion and health checks and you land in 26–49 s.
Any change must respect loop_wait + 2 * retry_timeout <= ttl, and ttl cannot go below 20.
Faster settings increase the risk of demoting a healthy primary on short DCS hiccups.
HAProxy check intervals and client reconnect behaviour matter as much as Patroni's timers.

Disclosure: I work on HighAvailabilityData, a free on-premise console from NUTRADELIS S.L. for Patroni clusters. Instead of only waiting for the TTL, it lets the HAProxy instances and the console vote: if a strict majority sees the primary down for lb_failover_after (10 s by default), it promotes the healthy replica with the most WAL applied through Patroni's POST /failover, releases the stale leader key with a compare-and-delete and puts the old primary in maintenance on every HAProxy. In our tests writes came back in about 15–20 s; measure in your own environment. The original article includes a step-by-step test plan.

Sources

Patroni, Dynamic Configuration Settings (loop_wait, ttl, retry_timeout, defaults, minimums and the loop_wait + 2*retry_timeout <= ttl rule): https://patroni.readthedocs.io/en/latest/dynamic_configuration.html
Patroni, REST API (GET /primary, POST /failover): https://patroni.readthedocs.io/en/latest/rest_api.html
Patroni, DCS Failsafe Mode (primary demoted when the leader lock update fails): https://patroni.readthedocs.io/en/latest/dcs_failsafe_mode.html
Patroni, Watchdog support: https://patroni.readthedocs.io/en/latest/watchdog.html
HAProxy configuration manual (inter, fall, rise, on-marked-down): https://docs.haproxy.org/
Pigsty documentation, Patroni passive failure RTO model (best/worst case per phase): https://pigsty.io/docs/concept/ha/failure/passive/
HighAvailabilityData, original article: https://www.highavailabilitydata.com/resources/failover-postgresql?lang=en

PostgreSQL high availability with only 2 servers: where the third vote comes from

Diego Gil Serrano — Sun, 25 Oct 2026 05:40:25 +0000

Many teams have budget for two database servers, not three. Almost every PostgreSQL high-availability guide, on the other hand, opens with "you need at least three nodes".

Both statements are true at the same time. You can run automatic failover with two database servers, but you cannot run it safely with two votes. This post explains why, and where the third vote can come from.

The question every failover has to answer

When a replica stops hearing from the primary, it cannot tell the difference between two situations:

The primary is down.
The network between the two nodes is down, and the primary is happily serving clients.

If the replica promotes itself in case 2, you have split-brain: two primaries accepting different writes. When the network comes back, those two histories cannot be merged automatically, and somebody loses data.

Consensus systems solve this with a majority rule: an action only happens if more than half of the voters agree. With two voters the majority is two. If they disagree, nobody has a majority and nothing happens. That is safe, but it also means no failover.

How Patroni and etcd use the majority

Patroni manages PostgreSQL on each node and stores cluster state in a distributed configuration store (DCS), most often etcd:

The primary holds a leader key in etcd and renews it on every HA loop, every loop_wait seconds (default 10).
The key expires after ttl seconds (default 30). Only when it expires can a replica take it and promote itself.
If the primary cannot update the leader key, Patroni demotes it. In the documentation's words: "In case the update of the leader lock fails, Postgres is immediately demoted and started as read-only." (Patroni's optional failsafe_mode relaxes this when the primary can still reach every other Patroni member.)

So Patroni itself never decides by local opinion: it delegates the "who is primary" decision to etcd. And etcd is a Raft cluster that needs a majority of its members to accept writes. The etcd FAQ gives the numbers:

etcd members	Majority	Failures tolerated
1	1	0
2	2	0
3	2	1
4	3	1
5	3	2

Two etcd members tolerate zero failures, exactly like one. If one etcd member is on each database server and one server dies, the survivor has 1 of 2 votes, etcd loses quorum, no leader key can be written, and the surviving node cannot become (or even stay) primary. The etcd FAQ also notes that adding a member to reach an even number "doesn't buy additional fault tolerance", which is why the usual recommendation is 3 or 5 members.

The constraint is on voters, not on PostgreSQL instances. That is the door to two-server designs.

Option 1: a third database server

Three nodes, each running PostgreSQL, Patroni and etcd. It is the simplest topology to reason about, and you get a second replica for reads or for synchronous replication. It is also the most expensive option, because the third machine needs the same disk and memory as the others.

Option 2: a witness that only runs etcd

A small VM that runs only etcd, no PostgreSQL and no Patroni. etcd is light, so a modest machine is enough. What matters is placement: the witness must not share a failure domain with either database server (different hypervisor host at a minimum, a different room or site if you have one).

A minimal sketch of the three-member etcd cluster (names and IPs are examples):

# On the witness (repeat on db1 and db2 with their own --name and IPs)
etcd --name witness \
  --initial-advertise-peer-urls http://10.0.0.13:2380 \
  --listen-peer-urls http://10.0.0.13:2380 \
  --listen-client-urls http://10.0.0.13:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://10.0.0.13:2379 \
  --initial-cluster db1=http://10.0.0.11:2380,db2=http://10.0.0.12:2380,witness=http://10.0.0.13:2380 \
  --initial-cluster-state new

And in each patroni.yml, point Patroni at all three:

etcd3:
  hosts: 10.0.0.11:2379,10.0.0.12:2379,10.0.0.13:2379

Use TLS for peer and client traffic in production; it is left out here for brevity.

Now if db1 dies, db2 and the witness still form a majority (2 of 3), the leader key expires, and db2 can take it. If the link between db1 and db2 breaks but both can reach the witness, etcd still has one consistent view and only one node holds the key.

Option 3: let the load balancers vote

Applications rarely connect to PostgreSQL directly. In the common Patroni setup they go through HAProxy, which polls each node's Patroni REST API: GET /primary returns HTTP 200 "only when the Patroni node is running as the primary with leader lock", and HAProxy sends writes to whichever node answers 200.

That makes the load balancers useful observers. They see the cluster from the application side. If most of them cannot reach the primary, the problem is not just the link between the two database nodes; clients are affected too. A design that counts load balancer opinions as votes can supply the third vote with machines you probably already run. This is not something Patroni does out of the box: you need a component that collects those observations, requires a strict majority, and then acts through Patroni (for example via its REST API) without ever bypassing etcd's view of who holds the leader key.

Things to get right with two database nodes

Synchronous vs asynchronous replication. With a single replica in strict synchronous mode, losing the replica stops writes on the primary. With asynchronous replication, a failover can lose the last committed transactions. Choose deliberately, and use Patroni's maximum_lag_on_failover to keep a far-behind replica out of the election.
The frozen primary. A primary that hangs (stuck disk, stopped process) without crashing keeps the leader key until it expires, and HAProxy will keep trying to send it writes. This is the scenario with the longest outage.
Fencing. After a failover, the old primary must not receive writes again until it has rejoined as a replica.
Test it. Kill the primary, freeze it with SIGSTOP, cut the network between the nodes, and measure how long writes stop. A failover you have never tested is an assumption.

Summary

Two database servers are fine. Two voters are not.
etcd needs 3 (or 5) members for fault tolerance; the third one does not have to run PostgreSQL.
Your options for the third vote: a third DB node, an etcd-only witness, or a voting scheme built on the load balancers' view.

Disclosure: I work on HighAvailabilityData, a free on-premise console from NUTRADELIS S.L. that deploys and operates Patroni + etcd + HAProxy clusters over SSH. With two PostgreSQL nodes it uses the HAProxy instances and the console itself as the missing voters instead of a third database machine. The nodes remain standard Patroni clusters. More details in the original version of this article.

Sources

Patroni, Dynamic Configuration Settings (loop_wait, ttl, retry_timeout, maximum_lag_on_failover): https://patroni.readthedocs.io/en/latest/dynamic_configuration.html
Patroni, REST API (GET /primary): https://patroni.readthedocs.io/en/latest/rest_api.html
Patroni, DCS Failsafe Mode (demotion when the leader lock cannot be updated): https://patroni.readthedocs.io/en/latest/dcs_failsafe_mode.html
etcd FAQ (cluster size, majority, failure tolerance, odd number of members): https://etcd.io/docs/next/faq/
HighAvailabilityData, original article: https://www.highavailabilitydata.com/resources/postgresql-high-availability-2-servers?lang=en

DEV Community: Diego Gil Serrano

What NIS2 Article 21 actually asks you to log, and how to do it with a self-hosted SIEM

Where logging appears in the Directive

Where it gets concrete: Implementing Regulation 2024/2690

Mapping requirements to real sources

A minimal self-hosted pipeline

Why self-hosting matters here

Checklist

Sources

Why Patroni failover takes 26–49 seconds with default settings (and how to make it faster)

The three settings that matter

What happens when the primary dies

Two important exceptions

How to make it faster

1. Shrink ttl, loop_wait and retry_timeout together

2. Tune the HAProxy health checks

3. Make clients reconnect quickly

4. Handle the frozen primary explicitly

5. Measure, do not assume

Summary

Sources

PostgreSQL high availability with only 2 servers: where the third vote comes from

The question every failover has to answer

How Patroni and etcd use the majority

Option 1: a third database server

Option 2: a witness that only runs etcd

Option 3: let the load balancers vote

Things to get right with two database nodes

Summary

Sources

1. Shrink `ttl`, `loop_wait` and `retry_timeout` together