DEV Community

Charles Givre
Charles Givre

Posted on • Originally published at gtkcyber.com

Detecting Network Service Discovery (T1046) with Python

Once an attacker has a foothold, the next move is almost always to look around: what hosts are reachable, what ports are open, what is worth attacking next. That is Network Service Discovery (T1046), and it is loud if you know what to measure. Scanning has a shape that normal traffic does not: one source touching many destinations and many ports, and getting refused a lot because most of what it probes is not listening.

You do not need to fingerprint nmap or match a scanner signature. Attackers swap tools and slow their scans to evade those. The fan-out and the failure rate are intrinsic to the technique, and both fall out of pandas aggregations over connection logs.

Where T1046 Shows Up

Zeek conn.log is ideal because it records conn_state, which tells you whether a connection completed or was refused. Netflow works too if you derive the failure signal from flags and byte counts. You want source, destination, destination port, and the connection state per record.

import pandas as pd

def load_zeek(path):
    cols = None
    with open(path) as f:
        for line in f:
            if line.startswith("#fields"):
                cols = line.strip().split("\t")[1:]
                break
    return pd.read_csv(path, sep="\t", comment="#", names=cols,
                       na_values=["-", "(empty)"])

conn = load_zeek("conn.log")  # id.orig_h, id.resp_h, id.resp_p, conn_state, proto
Enter fullscreen mode Exit fullscreen mode

Measure the Fan-Out

A scanner talks to far more distinct destinations and ports than a normal client. Group by source and count the spread:

fanout = conn.groupby("id.orig_h").agg(
    distinct_dsts=("id.resp_h", "nunique"),
    distinct_ports=("id.resp_p", "nunique"),
    connections=("id.resp_p", "count"),
)
Enter fullscreen mode Exit fullscreen mode

A workstation that talks to a handful of servers all day looks nothing like a host that opened connections to 200 distinct destinations. But fan-out alone flags busy infrastructure too (DNS resolvers, proxies, vulnerability scanners), so it is a lead, not a verdict.

Add the Failed-Connection Ratio

This is the feature that separates a scan from a busy server. Scanners hit closed and filtered ports constantly, so a large fraction of their connections never complete. In Zeek, those are states like S0 (no reply), REJ (refused), and RSTO. Compute the failure ratio per source and combine it with fan-out:

failed_states = ["S0", "REJ", "RSTO", "RSTOS0", "SH"]
conn["failed"] = conn["conn_state"].isin(failed_states)

fanout["fail_ratio"] = conn.groupby("id.orig_h")["failed"].mean()

scanners = fanout[
    (fanout["distinct_dsts"] >= 25) & (fanout["fail_ratio"] > 0.5)
].sort_values("distinct_dsts", ascending=False)
Enter fullscreen mode Exit fullscreen mode

High fan-out and a failure ratio above 50 percent is a strong network service discovery signal. A legitimate busy server has high connection counts but a low failure ratio, because the things it talks to are actually listening. The ratio is what cuts that false positive.

Horizontal vs. Vertical Scans

The two scan shapes need slightly different queries. A vertical scan hammers many ports on a few hosts; a horizontal scan sweeps one port across many hosts looking for a specific service (think SMB on 445 or RDP on 3389).

# Vertical: many ports, few destinations
vertical = fanout[(fanout["distinct_ports"] >= 50) & (fanout["distinct_dsts"] <= 3)]

# Horizontal: one port, many destinations
horizontal = (conn.groupby(["id.orig_h", "id.resp_p"])["id.resp_h"]
                   .nunique().reset_index(name="hosts"))
horizontal = horizontal[horizontal["hosts"] >= 25].sort_values("hosts", ascending=False)
Enter fullscreen mode Exit fullscreen mode

A horizontal sweep of port 445 across a /24 is reconnaissance for lateral movement, and it is worth an alert on its own.

Cutting the False Positives

A few sources scan legitimately, and you should know them by name:

  • Vulnerability scanners (Tenable, Qualys, Rapid7) and monitoring systems (Nagios, Zabbix) fan out by design. Allowlist their hosts.
  • Domain controllers and proxies show high connection counts but low failure ratios, so the ratio filter already separates them.
  • Slow scans spread the fan-out over hours to stay under per-minute thresholds. Run the aggregation over a longer window (a day, not a minute) and the spread still shows up, because the technique cannot avoid touching many destinations eventually.

The combination of wide fan-out, a high failure ratio, and a source that is not on your scanner allowlist is hard to explain as anything but discovery.

Catch the Shape, Not the Tool

Scan detection built on tool signatures breaks the moment someone writes a new scanner or slows the old one down. Fan-out and failure rate are properties of the technique itself, which is why measuring them holds up. That is the same principle behind the rest of this series, from beaconing to adversary-in-the-middle: detect the behavior, not the binary.

It is also what we teach in GTK Cyber's Threat Hunting with Data Science course. The T1046 reference page has the ATT&CK detail, and the threat hunting pipeline post shows how to schedule these queries instead of running them by hand.

Top comments (0)