Week 3: Introduction to John the Ripper

In your way through penetration testing, you will find yourself in situations where passwords or critical information are encoded. When this occurs, you first need to find in which format that hash is and then, try to decode it. For these cases, I like to use John the Ripper, one of the most popular password crackers around. So, let’s begin!

Formats

A format is just the kind of encoding that you’re trying to use. Let’s check how many formats john has by typing john --list=formats. You should see something like this:

j2rgez@myPC:~$ john --list=formats
descrypt, bsdicrypt, md5crypt, md5crypt-long, bcrypt, scrypt, LM, AFS,
tripcode, AndroidBackup, adxcrypt, agilekeychain, aix-ssha1, aix-ssha256,
aix-ssha512, andOTP, ansible, argon2, as400-des, as400-ssha1, asa-md5,
AxCrypt, AzureAD, BestCrypt, bfegg, Bitcoin, BitLocker, bitshares, Bitwarden,
BKS, Blackberry-ES10, WoWSRP, Blockchain, chap, Clipperz, cloudkeychain,
dynamic_n, cq, CRC32, sha1crypt, sha256crypt, sha512crypt, Citrix_NS10,
dahua, dashlane, diskcryptor, Django, django-scrypt, dmd5, dmg, dominosec,
dominosec8, DPAPImk, dragonfly3-32, dragonfly3-64, dragonfly4-32,
dragonfly4-64, Drupal7, eCryptfs, eigrp, electrum, EncFS, enpass, EPI,
EPiServer, ethereum, fde, Fortigate256, Fortigate, FormSpring, FVDE, geli,
gost, gpg, HAVAL-128-4, HAVAL-256-3, hdaa, hMailServer, hsrp, IKE, ipb2,
itunes-backup, iwork, KeePass, keychain, keyring, keystore, known_hosts,
krb4, krb5, krb5asrep, krb5pa-sha1, krb5tgs, krb5-17, krb5-18, krb5-3,
kwallet, lp, lpcli, leet, lotus5, lotus85, LUKS, MD2, mdc2, MediaWiki,
monero, money, MongoDB, scram, Mozilla, mscash, mscash2, MSCHAPv2,
mschapv2-naive, krb5pa-md5, mssql, mssql05, mssql12, multibit, mysqlna,
mysql-sha1, mysql, net-ah, nethalflm, netlm, netlmv2, net-md5, netntlmv2,
netntlm, netntlm-naive, net-sha1, nk, notes, md5ns, nsec3, NT, o10glogon,
o3logon, o5logon, ODF, Office, oldoffice, OpenBSD-SoftRAID, openssl-enc,
oracle, oracle11, Oracle12C, osc, ospf, Padlock, Palshop, Panama,
PBKDF2-HMAC-MD4, PBKDF2-HMAC-MD5, PBKDF2-HMAC-SHA1, PBKDF2-HMAC-SHA256,
PBKDF2-HMAC-SHA512, PDF, PEM, pfx, pgpdisk, pgpsda, pgpwde, phpass, PHPS,
PHPS2, pix-md5, PKZIP, po, postgres, PST, PuTTY, pwsafe, qnx, RACF,
RACF-KDFAES, radius, RAdmin, RAKP, rar, RAR5, Raw-SHA512, Raw-Blake2,
Raw-Keccak, Raw-Keccak-256, Raw-MD4, Raw-MD5, Raw-MD5u, Raw-SHA1,
Raw-SHA1-AxCrypt, Raw-SHA1-Linkedin, Raw-SHA224, Raw-SHA256, Raw-SHA3,
Raw-SHA384, ripemd-128, ripemd-160, rsvp, Siemens-S7, Salted-SHA1, SSHA512,
sapb, sapg, saph, sappse, securezip, 7z, Signal, SIP, skein-256, skein-512,
skey, SL3, Snefru-128, Snefru-256, LastPass, SNMP, solarwinds, SSH, sspr,
Stribog-256, Stribog-512, STRIP, SunMD5, SybaseASE, Sybase-PROP, tacacs-plus,
tcp-md5, telegram, tezos, Tiger, tc_aes_xts, tc_ripemd160, tc_ripemd160boot,
tc_sha512, tc_whirlpool, vdi, OpenVMS, vmx, VNC, vtp, wbb3, whirlpool,
whirlpool0, whirlpool1, wpapsk, wpapsk-pmk, xmpp-scram, xsha, xsha512, ZIP,
ZipMonster, plaintext, has-160, HMAC-MD5, HMAC-SHA1, HMAC-SHA224,
HMAC-SHA256, HMAC-SHA384, HMAC-SHA512, dummy, crypt

Wow, that’s a lot, right? Well, names and/or well-known formats (like MD5 which starts with $1$), could lead you to know which format you’re going to use. So let’s create a new file called my_hash.txt and insert the following hash (in user:password format): root:$1$6ff3402b$2w6aUd7n//XodMXDt84BE1.

Now, to select the decoded format, we type john my_hash.txt --format=md5crypt and check the results:

j2rgez@myPC:~$ john my_hash.txt --format=md5crypt
Using default input encoding: UTF-8
Loaded 1 password hash (md5crypt, crypt(3) $1$ (and variants) [MD5 256/256AVX28x3])
Will run 12 OpenMP threads
Proceeding with single, rules:Single
Press 'q' or Ctrl-C to abort, almost any other key for status
Almost done: Processing the remaining buffered candidate passwords, if any.
Warning: Only 248 candidates buffered for the current salt, minimum 288 needed for performance.
Proceeding with wordlist:/usr/share/john/password.lst, rules:Wordlist
admin            (root)
1g 0:00:00:00 DONE 2/3 (2020-04-02 20:38) 20.00g/s 90560p/s 90560c/s 90560C/s chacha..OU812
Use the "--show" option to display all of the cracked passwords reliably
Session completed       

As you can see, john tried to find a match in its default wordlist and found it! The decoded value is “admin” (let's keep things fast).

There are tons of wordlists around the internet (or you can create your own ones). You can check for example the ones from SecLists repo, and simply type john my_hash.txt --format=md5crypt --wordlist=my_wordlist.txt.

Automating all of this

So, last weekend I was completing some exercises from Pentesterlab when I found this sentence in one of the exercises: Use John by trial and error until you find the format. From the exercise context, you could imagine which format it was, but I was thinking: what if they ask me to do this in the future with a not so obvious format?

So I began my way of automating pentesting processes and created a new repo. Yes, I’m 90% sure that there is a better way (or an existing tool) to automate John the Ripper (or password cracking in general), but I find that it’s also a good way to practice some new programming languages or at least the ones that I don’t use on a day to day basis. So here is the first version of my john_auto_decypher.py script:

import subprocess
import argparse

def decrypt(formats, inputFile):
    keep_string = "Press 'q' or Ctrl-C to abort, almost any other key for status"
    format = ""
    for f in formats:
        try:
            decode_process = subprocess.check_output(["john", inputFile, "--format=" + f], stdout=None)
            if keep_string in decode_process:
                format = f
        except: subprocess.CalledProcessError

    if format:
        print "Found a hash for " + format + " format"
    else:
        print "Couldn't decode, checking for already saved hashes..."

    print subprocess.check_output(["john", "--show", inputFile])

def getArgs():
    parser = argparse.ArgumentParser("john_auto_decypher.py")
    parser.add_argument("file", help="File with encoded passwords to be cracked", type=str)
    return parser.parse_args()

def main():
    inputFile = getArgs().file
    formats = map(lambda str: str.strip(), subprocess.check_output(["john", "--list=formats"]).split(","))
    decrypt(formats, inputFile)

main()

Simple but effective, now by just typing python john_auto_decypher.py <filename> on my CLI, this script will check all formats for me and/or already saved hashes. Of course, there is still a lot to improve, like adding parameters for wordlists and custom rules.

So, that's all for today. Have a very good weekend you all!

Previous weeks:
Week 0: Security blog posts introduction
Week 1: Being conscious about your attack surface
Week 2: Knowing the penetration testing steps