Hello, Worm!: Mapping SSH probes with a bash script

Please note this is meant to be more fun than serious security advice

Contributing to TLDR Pages has introduced me to tons of interesting and useful commands like lastb, which shows the details of failed login attempts. Of course you can't write good instructions for unfamiliar material so I ran lastb -iw on one of my AWS VMs and got a rather unpleasant surprise:

AWS_VM: sudo lastb -iw | wc -l 
967

967 failed logins? I know this wasn't me because I authorize with public keys. I doubt that this is a targeted attack because this VM doesn't have an aftermarket CNAME or publicized IP address. Are there really this many botnets in-flight?

The nice people from GCP offer a no-strings micro VM for free, let's provision one and see how long it sits unmolested before it's probed for SSH

GCP_VM: uptime
22:21:47 up 5 min,  1 user,  load average: 0.00, 0.00, 0.00
GCP_VM: lastb | wc -l 
2

Well this is a grim development: 5 minutes until the first SSH probe. I would like to see where these people are coming from, but don't want to waste my (very) limited resources on a dedicated honeypot; let's try for lightweight data collection on already-running VMs.

Choose Your Weapon

• bash and awk can be considered part of the Linux firmament.

• curl is pretty ubiquitous on interactive VMs.

• https://mapbox.io graciously offers an SVG mapping API with a free-tier.

• https://ipstack.com offers an IP address geolocation API with a free-tier.

To avoid overwhelming the goodwill of these API providers, we will restrict our queries to IPs that probed our VM in the last 24 hours.

#!/bin/bash

# Set token data here. Because requests want a token in the URL,
# a .netrc file can't be used to store credentials
#
IPSTACK_TOKEN=""
MAPBOX_TOKEN=""

# Get a timestamp of 24 hours ago in epoch seconds
#
STANDOFF=$(date -d "24 hours ago" +%s)

# Capture failed logins with a timestamp < 24-hours-ago to a buffer file
#
sudo lastb -i | awk '{ print $3,$5,$6,$7 }' | while read ip datum
do
  [[ "$(date -d "$datum" +%s)" -gt "$STANDOFF" ]] && echo -e "$ip"
done > /tmp/LASTB_IP_BUFFER

# Get the top 150 IPs by connection count
sort /tmp/LASTB_IP_BUFFER | uniq -c | sort -rnk1 | awk '{ print $2,$1 }' | head -150 | while read IP time
do
    # Get Lat/Long data
    curl -s "http://api.ipstack.com/$IP?access_key=$IPSTACK_TOKEN&fields=longitude,latitude" | python -m json.tool | awk '/longitude|latitude/ { printf $NF" " }' | tr -d "," | awk '$1 != "null" { print "pin-s+0FF("$2","$1")," }'

# Smush all coordinates into a string to make a single call to Mapbox.io
done | tr -d "\n" | sed -e 's/,$//g' > /tmp/coords

curl -s "https://api.mapbox.com/styles/v1/mapbox/dark-v10/static/"$(cat /tmp/coords)"/0,40,1.35,0/1280x1024?access_token=$MAPBOX_TOKEN" > worms.png

rm /tmp/LASTB_IP_BUFFER /tmp/coords

And here we have a handsome world map with pins stuck in every probing botnet's source-IP location, suitable for framing or scaring your boss.

Comments

[Thomas H Jones II]

I run fail2ban on any internet-facing systems I'm responsible for. This article mad me curious, so I did a quick scan of my failed logins log (on my personal VPS). Results are pretty grim:

• logrotate had rotated the log earlier today ...so the following numbers are < 24 hours
• 5732 attempts
• 876 unique userids (used tr to convert them all to lowercase then ran that list through uniq)
• 40,241 login failures cataloged by fail2ban
• 12,433 IPs banned
• 55 IPs within the ban-rotation window (1 hour for the ssh service, specifically)

The fail2ban stuff gets even more grim when extended to SMTP

[Denis Rasulev]

+1 for fail2ban! Works like a charm. I've added ip-set to it lately and this has helped to reduce load significantly.

[Thomas H Jones II]

Yeah. ip-set rules are great for ensuring across-boot persistence, too.

One of these days, I'll get around to integrating my deployment-configuration with a "phone home" hook that informs the configuration service, "when re-provisioning this host or provisioning new hosts, blacklist these IPs".

[lbonanomi]

Thank you @bhilburn for the kind words and everyone for the warm reception!

[astodev]

Looks like mapbox is down for now.. :/

[Thomas H Jones II]

Unfortunately, as a consultant, I initiate connections from a wide variety of locations. Some of those locations block "weird" ports. So, moving to non-default port is generally not an option for me.