Python query IP address range and get Hostname

Recently, LinkedIn's web crawlers have been extensively scraping websites. To identify which of these are LinkedIn's crawlers, we performed reverse IP address lookups and determined the hostnames.

After identifying the IP address ranges, we needed to confirm if each IP's hostname matches 'x.fwd.linkedin.com'.

Below is a Python code example for determining the hostname of each IP address:

import socket

# Define a list of IP addresses to check
start_ip = "108.174.2.0"
end_ip = "108.174.255.255"

# Convert the IP address string to an integer for iteration within the range.
def ip_to_int(ip):
    parts = ip.split(".")
    return (int(parts[0]) << 24) + (int(parts[1]) << 16) + (int(parts[2]) << 8) + int(parts[3])

start_ip_int = ip_to_int(start_ip)
end_ip_int = ip_to_int(end_ip)

# Perform a reverse DNS lookup to get the hostname
for ip_int in range(start_ip_int, end_ip_int + 1):
    ip = ".".join(str((ip_int >> i) & 255) for i in (24, 16, 8, 0))  # Convert the integer back to an IP address string.
    try:
        hostname, _ = socket.gethostbyaddr(ip)
        print(f"IP: {ip} -> Hostname: {hostname}")
    except socket.herror:
        # If the hostname cannot be resolved, print "N/A."
        print(f"IP: {ip} -> Hostname: N/A")

This way, you can determine which IP addresses belong to LinkedIn's web crawlers.