The scenario below is fictional (all IPs are RFC 5737 documentation ranges).
The worst moment in an investigation is realizing the logs are gone. It's tempting to conclude the attacker was better than you. That's only half true: the act of erasing evidence leaves new evidence.
This is part 2 of a series. Part 1 showed one micro-technique (recovering a deleted binary from /proc). Here is the map: seven principles every later part fits into.
What you'll learn
- Which traces attackers can erase, and which they can't
- Seven principles for reading traces without getting lost
- A 3-question audit you can run on your own servers
Four kinds of traces
| Kind | Examples | Can the attacker erase it? |
|---|---|---|
| Self-reported |
.bash_history, app logs |
Easily |
| Runtime state |
/proc/<PID>/, live connections |
Not without stopping the process |
| Kernel/OS records | auditd, Windows event logs, file ctime | Needs privilege, and erasing leaves a record |
| Off-host records | central logs, firewall/flow logs, cloud logs | Out of reach |
The seven principles
- Read traces from the attacker's point of view. Foothold, persistence, lateral movement, cleanup: each step has predictable places to leave marks.
- Know normal, then diff. Detection is mostly "difference from baseline", not "spot the weird".
- Traces are redundant. One action shadows several places. If one is wiped, another corroborates.
- Deleted and silent are evidence. A log that stops is itself data. Report "no logs" as unverified, never as safe.
- Normalize to UTC, build one timeline.
- Separate fact from inference. "ctime is 02:23" is a fact. "They got in at 02:23" is a hypothesis.
- Don't stop at detection. Feed findings back into design: central logging, immutability, least privilege.
Three questions for tomorrow
- Are execution records sent somewhere the server itself can't erase?
- Can you describe your "normal admin path" (bastion, accounts, maintenance window) in writing?
- If a log stops, will anyone notice?
Takeaways
Attackers can erase their own diary, but not the OS's and the network's records. Next: a file backdated to 2018 with touch, caught with one line.
Adapted from chapter 1 of my Japanese book on server defense (40 chapters, Windows and Linux, MITRE ATT&CK-mapped, with a laptop-sized lab).
Top comments (0)