DEV Community

Cover image for On the wire: network traffic analysis and detection engineering with open tools
Marcin
Marcin

Posted on

On the wire: network traffic analysis and detection engineering with open tools

#suricata #tcpdump #tshark #tutorial

Most write-ups about network security tools stop at "here are the flags." That is the least interesting part. The interesting part is the reasoning: you have a capture or a live sensor, you have a question — did this host get owned, and how would I know? — and you need to turn a wall of packets into an answer you can defend.

This article walks through the toolchain I actually use for that, in the order I usually reach for it: capture and triage with tcpdump and tshark, flow-level context with SiLK, protocol reconstruction with Zeek, signature detection with Snort and Suricata, and packet crafting with Scapy to test your own rules. The second half is dedicated to writing detection rules, from a one-line signature to entropy-aware DNS tunneling checks.

Every command here is copy-paste ready. The companion cheat sheets for each tool live in the same repository.

The toolkit at a glance

Each tool answers a different question. Reaching for the wrong one is how you burn an afternoon.

Tool The question it answers
tcpdump "Show me packets matching this precisely."
tshark "Decode and extract fields from a protocol."
SiLK "Who talked to whom, how much, over what period?"
Zeek "Give me structured logs of everything that happened."
Snort / Suricata "Alert me when traffic matches a known-bad pattern."
Scapy "Let me build a packet by hand and send it."

Capture is the raw material. Flow data is the map. Zeek logs are the timeline. Signatures are the verdict. Scapy is your test harness. Keep those roles separate in your head and the workflow falls into place.

Part 1 — Capture and first triage

tcpdump is the tool you never fully outgrow. Its power is the Berkeley Packet Filter (BPF) language: you can express, before a single packet is decoded, exactly what you want to see.

Start simple — read a file, skip name resolution (which is slow and can leak lookups), and drop timestamps for cleaner grep-able output:

tcpdump -r capture.pcap -n
tcpdump -r capture.pcap -ntc 20        # first 20 records only
tcpdump -r capture.pcap -nX 'dst host 10.10.10.5 and src port 4444'
Enter fullscreen mode Exit fullscreen mode

The -X flag prints hex and ASCII side by side, which is enough to eyeball a plaintext C2 beacon or a suspicious user agent without opening Wireshark.

Filtering on the bits, not just the fields

Where BPF earns its keep is reaching into header bytes directly. The TCP flags live in a single byte, tcp[13]:

Bit 7 6 5 4 3 2 1 0
Flag CWR ECE URG ACK PSH RST SYN FIN
Hex 0x80 0x40 0x20 0x10 0x08 0x04 0x02 0x01

To find connection attempts — SYN set, ACK not set — mask off the two high bits and compare:

tcpdump -r capture.pcap -n 'tcp[13] & 0x3f = 0x02'
Enter fullscreen mode Exit fullscreen mode

Chain that into a quick top-ports summary and you have a horizontal-scan detector in one line:

tcpdump -r capture.pcap -n 'tcp[13] & 0x3f = 0x02' \
  | cut -d ' ' -f 5 | cut -d . -f 5 | sort | uniq -c | sort -nr | head
Enter fullscreen mode Exit fullscreen mode

The same byte-offset trick works on DNS. A DNS message rides in the UDP payload, and since the UDP header is 8 bytes, the DNS header starts at udp[8]. That fixes the offsets you care about:

DNS field UDP offset
Flags udp[10:2]
Questions (QDCOUNT) udp[12:2]
Answers (ANCOUNT) udp[14:2]
Authority (NSCOUNT) udp[16:2]
Additional (ARCOUNT) udp[18:2]

The high bit of udp[10] is the QR flag: 0 for a query, 1 for a response. So "all DNS queries" and "all DNS responses" are:

tcpdump -r dns.pcap -n 'dst port 53 and udp[10] & 0x80 = 0'      # queries
tcpdump -r dns.pcap -n 'src port 53 and udp[10] & 0x80 = 0x80'   # responses
Enter fullscreen mode Exit fullscreen mode

This is worth doing by hand once, even though Zeek and Suricata will parse DNS for you later. Understanding where the bits are is what lets you write good rules — and debug the ones that do not fire.

tshark when you need the parsed field

tcpdump filters on raw structure. tshark understands protocols, so you can pull named fields straight into columns — perfect for feeding a spreadsheet or another script:

tshark -r dns.pcap -n -Y 'udp.port == 53' \
  -T fields -e ip.src -e ip.dst -e dns.qry.name -e dns.a
Enter fullscreen mode Exit fullscreen mode

Two tshark features I use constantly: stream numbers to isolate a conversation, and follow to dump one stream as text:

tshark -r capture.pcap -n -Y 'tcp.port == 25' -T fields -e tcp.stream | uniq
tshark -r capture.pcap -n -q -z follow,tcp,ascii,9 | less
Enter fullscreen mode Exit fullscreen mode

Part 2 — Flow-level context with SiLK

Packets are detail; flows are perspective. When you are staring at gigabytes and do not yet know what matters, NetFlow tells you who talked to whom, how much, and when — without the payload.

SiLK's rwfilter selects flows and pipes them to rwcut for display:

rwfilter --type=all --start 2024/01/01 --end 2024/12/31 \
  --proto=6 --dport=443 --pass=stdout \
  | rwcut --fields=sip,sport,dip,dport,bytes --no-columns
Enter fullscreen mode Exit fullscreen mode

The pattern that pays off in an investigation is chaining filters to answer a real question — for example, "what leaves my internal subnets but does not land in another internal subnet?" (i.e. traffic actually heading out):

rwfilter --type=all --start 2024/01/01 --end 2024/12/31 \
  --scidr=10.200.0.0/16 --proto=6 --pass=stdout \
  | rwfilter --dcidr=10.200.0.0/16 --fail=stdout --input-pipe=stdin \
  | rwcut --fields=sip,sport,dip,dport,bytes,flags --no-columns
Enter fullscreen mode Exit fullscreen mode

The trick is --input-pipe=stdin on the second rwfilter, and --fail to keep only the flows that did not match the internal destination. For top-talker summaries, rwstats does the counting for you.

Part 3 — Turning packets into evidence with Zeek

Zeek is the tool that changed how I read traffic. Instead of decoding packets, it produces structured, tab-separated logs — conn.log, dns.log, http.log, ssl.log, files.log, x509.log — and stitches them together with a shared connection id (uid). Find one interesting record and you can pivot to every other log entry for the same connection.

zeek -r capture.pcap                          # generates *.log in the cwd
cat http.log | zeek-cut -u ts uid id.orig_h id.resp_h host uri status_code
Enter fullscreen mode Exit fullscreen mode

zeek-cut extracts named columns (the -u renders timestamps in UTC). Chasing a file by its id across every log is a one-liner:

grep FYAp0L1VyrDDhX8uu *.log
Enter fullscreen mode Exit fullscreen mode

Beyond logging, Zeek is programmable. You can write scripts that react to events — extract every file carried over HTTP, flag a DNS answer that points at a domain you saw in an email, and so on. That scripting layer is where Zeek stops being a logger and becomes a detection platform, but even the default logs alone are worth the setup.

Part 4 — Signature detection: Snort and Suricata

Snort and Suricata both match traffic against rules and raise alerts. Suricata is multi-threaded and speaks the richer, more modern rule syntax, so the rule-writing section below uses it — but the two dialects are close cousins.

Run Suricata over a pcap and point it at your ruleset:

suricata -T -c suricata.yaml -S local.rules -vvv        # validate config + rules
suricata -r sample.pcap -c suricata.yaml -S local.rules -l ./logs/
Enter fullscreen mode Exit fullscreen mode

Alerts and protocol events land in logs/eve.json in EVE format — JSON, one event per line, which means jq is your query engine:

jq -c 'select(.event_type=="alert") | {sig: .alert.signature, src: .src_ip, dst: .dest_ip}' logs/eve.json
Enter fullscreen mode Exit fullscreen mode

Note that the signature id is nested under the alert object as .alert.signature_id, not at the top level — a small detail that quietly returns null if you get it wrong.

Snort's equivalents are worth knowing when you land on a Snort box:

snort -r sample.pcap -c snort.lua -q -A alert_fast -R local.rules
Enter fullscreen mode Exit fullscreen mode

Part 5 — Writing detection rules, simple to advanced

This is the part that separates using an IDS from operating one. We will start from a rule you can read at a glance and finish with stateful, intel-backed detection that reasons about byte layout and entropy. Along the way, the rules pick up the scaffolding that distinguishes a lab example from something you would actually deploy: precise buffers, a fast-pattern anchor, MITRE-tagged metadata, tuning, and a lifecycle.

Anatomy of a rule

Every Suricata rule has three parts: an action, a header, and options in parentheses.

action protocol src_ip src_port -> dst_ip dst_port (options)
Enter fullscreen mode Exit fullscreen mode
  • Action — alert (log it), drop (block, IPS mode), reject (block and tear down), pass (allow, stop evaluating).
  • Header — the protocol (tcp, udp, dns, http, tls, …), addresses and ports, and direction (-> or <>). $HOME_NET and $EXTERNAL_NET are variables defined in suricata.yaml.
  • Options — the detection logic plus the metadata that makes a rule maintainable.

A rule with no msg, no classtype, no reference, and no metadata will still fire — and it will be worthless six months later when an analyst triages the alert and has no idea what it means or how confident to be. Treat metadata as part of the detection, not decoration.

Level 1 — a precise, documented first rule

The textbook version of a first rule is "match a bad hostname." The deployable version scopes the flow, anchors the fast pattern, pins the buffer length, and carries enough context to triage:

alert http $HOME_NET any -> $EXTERNAL_NET any ( \
    msg:"MALWARE Outbound HTTP to known-bad C2 host"; \
    flow:established,to_server; \
    http.host; content:"evil.example.com"; fast_pattern; bsize:16; \
    classtype:trojan-activity; \
    reference:url,attack.mitre.org/techniques/T1071/001/; \
    metadata:attack_target Client_Endpoint, deployment Perimeter, \
      signature_severity Major, confidence High, \
      mitre_tactic_id TA0011, mitre_technique_id T1071, \
      created_at 2024_01_10, updated_at 2024_01_10; \
    sid:1000001; rev:1;)
Enter fullscreen mode Exit fullscreen mode

What earns each keyword its place:

  • flow:established,to_server — evaluate only the client-to-server side of an established session. This one keyword removes most false positives and a lot of wasted CPU.
  • bsize:16 — the http.host buffer must be exactly 16 bytes (evil.example.com), so a lookalike subdomain does not slip through and the match is unambiguous.
  • fast_pattern — tells the engine which content to load into the multi-pattern matcher for the first-pass filter. On a busy sensor, choosing a good fast pattern is the single biggest performance lever you have.
  • classtype / reference / metadata — the triage context: what kind of activity, where it maps in ATT&CK, how severe, and how much to trust it. Keep local sids in the 1000000+ range and bump rev on every edit.

Level 2 — content matching that performs

content is the workhorse; the modifiers are what make it precise and cheap. The goal is always to match the smallest, most specific buffer possible.

http.method; content:"POST";
http.uri; content:".php?id="; fast_pattern;
http.user_agent; content:"Mozilla/4.0 (compatible|3b 20|MSIE";
http.header_names; content:!"Referer"; content:!"Accept-Language";
Enter fullscreen mode Exit fullscreen mode
  • offset / depth constrain where to look; distance / within position one content relative to the previous one — enough to parse structured payloads without paying for a regex.
  • Negation (content:!"...") is underused: a lot of automated clients give themselves away by the headers a real browser sends and they omit. Matching on http.header_names for absent Referer and Accept-Language is a classic, cheap C2 heuristic.
  • Bytes you cannot type go in pipes as hex: |3b 20| is "; ".
  • Put fast_pattern on the longest, rarest content — never on something like "GET" that appears in nearly every packet.

Level 3 — stateful detection: flowbits and xbits

Single-packet rules miss anything that unfolds over a conversation. flowbits carries state within a flow; xbits carries it across flows on the same host, which is how you correlate, say, a phishing click with a later beacon from the same endpoint.

Two-stage detection within one flow — a download request followed by an executable in the response — alerting only on the combination:

alert http $HOME_NET any -> $EXTERNAL_NET any ( \
    msg:"MALWARE Stage 1 suspicious payload request"; \
    flow:established,to_server; \
    http.uri; content:"/api/v1/getfile"; fast_pattern; \
    flowbits:set,payload.request; flowbits:noalert; \
    classtype:trojan-activity; sid:1000010; rev:1;)

alert http $EXTERNAL_NET any -> $HOME_NET any ( \
    msg:"MALWARE PE delivered after suspicious request"; \
    flow:established,to_client; \
    flowbits:isset,payload.request; \
    file.data; content:"MZ"; startswith; \
    content:"This program cannot be run in DOS mode"; distance:0; \
    classtype:trojan-activity; \
    metadata:mitre_tactic_id TA0011, mitre_technique_id T1105; \
    sid:1000011; rev:1;)
Enter fullscreen mode Exit fullscreen mode

The first rule sets a bit and stays silent (noalert); the second fires only if that bit is set earlier in the same flow and the response actually carries a PE. Correlating across flows on a host looks the same, with xbits:set,name, track ip_src, expire 3600; and a matching xbits:isset.

Level 4 — protocol intelligence (DNS and TLS)

Modern detection lives in the parsed protocol buffers, and it shines on traffic you cannot read.

DNS — beaconing and tunneling. A single long query means little; many long, high-entropy queries to one parent domain in a short window is what exfiltration looks like. Combine a PCRE length/charset test with a threshold so you alert on the behavior, not on each packet:

alert dns $HOME_NET any -> any any ( \
    msg:"MALWARE Possible DNS tunneling - sustained long high-entropy labels"; \
    dns.query; \
    pcre:"/^[a-z0-9]{40,}\.[a-z0-9\-]+\.[a-z]{2,}$/i"; \
    threshold:type both, track by_src, count 15, seconds 60; \
    classtype:bad-unknown; \
    reference:url,attack.mitre.org/techniques/T1071/004/; \
    metadata:attack_target Client_Endpoint, deployment Perimeter, \
      signature_severity Major, confidence Medium, \
      mitre_tactic_id TA0011, mitre_technique_id T1071, \
      created_at 2024_01_10, updated_at 2024_01_10; \
    sid:1000020; rev:1;)
Enter fullscreen mode Exit fullscreen mode

threshold:type both, count 15, seconds 60 means: only alert after 15 matches from the same source within a minute, then at most once per window. That is the line between a signal and an alert storm. Pair it with hunting for a burst of NXDOMAIN responses in the DNS logs, which DGA and tunneling both generate heavily.

When you do want an exact domain — say, matching a C2 domain but not a lookalike subdomain of it — two keywords make it airtight. bsize pins the buffer length, and dotprefix prepends a . to the query buffer so an endswith cannot be fooled by evilcdn.example.com when you meant cdn.example.com:

alert dns $HOME_NET any -> any any ( \
    msg:"MALWARE C2 domain lookup"; \
    dns.query; dotprefix; content:".cdn.example.com"; endswith; \
    classtype:trojan-activity; sid:1000021; rev:1;)
Enter fullscreen mode Exit fullscreen mode

TLS — fingerprint what you cannot decrypt. The payload is encrypted, but the handshake is not: the SNI, the certificate subject/issuer, and the JA3/JA3S fingerprints are all in the clear and remarkably identifying (JA3 must be enabled under the TLS parser in suricata.yaml).

alert tls $HOME_NET any -> $EXTERNAL_NET any ( \
    msg:"MALWARE JA3 fingerprint matches known C2 client"; \
    ja3.hash; content:"a0e9f5d64349fb13191bc781f81f42e1"; \
    flow:established,to_server; \
    classtype:trojan-activity; \
    metadata:mitre_tactic_id TA0011, mitre_technique_id T1573, \
      signature_severity Major, confidence High; \
    sid:1000030; rev:1;)
Enter fullscreen mode Exit fullscreen mode

At scale you do not write one rule per indicator. A dataset matches a buffer against an external list and stays fast with tens of thousands of entries — the right structure for threat-intel feeds of bad JA3 hashes or domains:

alert tls $HOME_NET any -> $EXTERNAL_NET any ( \
    msg:"MALWARE TLS JA3 on threat-intel list"; \
    ja3.hash; dataset:isset,bad_ja3, type string, load bad_ja3.lst; \
    flow:established,to_server; \
    classtype:trojan-activity; sid:1000031; rev:1;)
Enter fullscreen mode Exit fullscreen mode

Level 5 — byte math for protocol and exploit detection

When detection depends on a value the protocol computes — a length field, a record count, an opcode — you need arithmetic on raw bytes. byte_extract pulls a value into a variable, byte_test compares against one, and byte_jump moves the cursor by a computed amount.

A generic "declared length is absurdly large" check — the shape of many buffer-overflow attempts against a length-prefixed protocol:

alert tcp $EXTERNAL_NET any -> $HOME_NET 1521 ( \
    msg:"EXPLOIT Oversized declared length field (possible overflow)"; \
    flow:established,to_server; \
    content:"|00 03|"; offset:0; depth:2; \
    byte_test:2, >, 4096, 2; \
    classtype:attempted-admin; \
    metadata:mitre_tactic_id TA0001, mitre_technique_id T1190; \
    sid:1000040; rev:1;)
Enter fullscreen mode Exit fullscreen mode

Read byte_test:2, >, 4096, 2 as: take the 2-byte value at offset 2 and match if it is greater than 4096. Swap the operator for & to test individual flag bits (byte_test:1, &, 0x80, 3 matches when the high bit of the byte at offset 3 is set).

byte_jump is the keyword that makes length-prefixed protocols matchable: read a length field, then skip exactly that many bytes so the cursor lands on whatever comes after a variable-length blob. Consider a protocol shaped as [magic][2-byte length][payload of that length][command byte] — you want the command byte, but you cannot know its offset in advance:

alert tcp $HOME_NET any -> $EXTERNAL_NET any ( \
    msg:"MALWARE C2 command after variable-length field"; \
    flow:established,to_server; \
    content:"|aa bb|"; depth:2; \
    byte_jump:2, 0, relative; \
    content:"|01|"; distance:0; within:1; \
    classtype:trojan-activity; sid:1000042; rev:1;)
Enter fullscreen mode Exit fullscreen mode

Step through it: content:"|aa bb|" matches the magic at the start and leaves the cursor at byte 2. byte_jump:2, 0, relative reads a 2-byte value at the cursor (the length field) and advances the cursor forward by that many bytes — over the whole variable payload. The final content:"|01|"; distance:0; within:1 then checks the single byte the cursor now points at. One rule, correct regardless of how long the payload is. Add multiplier N when the length is counted in words rather than bytes, or from_beginning to jump from the start of the buffer instead of the current position.

For proprietary binary C2 with no protocol parser to lean on, you match the wire bytes directly — and dsize (payload length) is a cheap, discriminating pre-filter. A fixed-size command header is a good example:

alert tcp $HOME_NET any -> $EXTERNAL_NET any ( \
    msg:"MALWARE Custom C2 fixed-size beacon"; \
    flow:established,to_server; dsize:16; \
    content:"|de ad be ef|"; depth:4; \
    content:"|01|"; distance:3; within:1; \
    classtype:trojan-activity; sid:1000041; rev:1;)
Enter fullscreen mode Exit fullscreen mode

dsize:16 throws out everything that is not exactly a 16-byte payload before any content match runs — the kind of early bail-out that keeps a sensor fast.

Tuning out false positives

This is the discipline that separates a rule you wrote from a rule you can run. The pattern is always the same: keep the true positives, carve out the benign traffic that happens to look similar — without gutting the rule into uselessness.

The first tool is negation. If a legitimate host or client trips your hunting rule, exclude precisely that, and nothing more:

# Base hunting rule — a generic C2 gate, but a legit app also posts to /gate.php
alert http $HOME_NET any -> $EXTERNAL_NET any ( \
    msg:"HUNT Request to /gate.php"; \
    flow:established,to_server; \
    http.uri; content:"/gate.php"; \
    classtype:bad-unknown; sid:1000050; rev:1;)

# Same rule, minus the one benign app — note the rev bump
alert http $HOME_NET any -> $EXTERNAL_NET any ( \
    msg:"HUNT Request to /gate.php"; \
    flow:established,to_server; \
    http.uri; content:"/gate.php"; \
    http.host; content:!"updates.legit-vendor.com"; \
    classtype:bad-unknown; sid:1000050; rev:2;)
Enter fullscreen mode Exit fullscreen mode

You can exclude on any buffer — a specific header the benign client sends (http.header; content:!"X-Requested-With: com.legit.app";), or the absence of one the malware never sets (http.header_names; content:!"User-Agent";).

The second tool is tightening the anchor. A short match at the wrong offset produces phantom hits. Suppose content:"Gh0st"; offset:8; depth:5; false-positives on unrelated traffic that happens to contain that string. Pin it to the byte that actually precedes it in the real protocol:

content:"|00|Gh0st"; offset:7; depth:6;
Enter fullscreen mode Exit fullscreen mode

Same string, but now it only matches where a null byte sits immediately before it — the structural detail the coincidental traffic lacks. Every false positive is a lesson about a byte you were not yet constraining.

Making rules production-ready

A rule that matches is only half the job. Before it goes live:

  • Order for the early bail-out. Suricata evaluates cheap checks before expensive ones, so give it a reason to quit early: lead with dsize, flow, and a long content before any pcre. Never ship a regex-only rule — always pair pcre with at least one content the engine can reject on first. Profiling (--enable-profiling) shows you which rules are actually eating CPU.
  • Pick the fast pattern deliberately. Run suricata --engine-analysis and read rules_analysis.txt — it tells you which content Suricata chose as the fast pattern and warns about rules with weak or no anchor. Set fast_pattern on the longest, rarest content; a bad fast pattern is the usual cause of a sensor falling behind.
  • Tune, do not silence. If a rule is noisy, reach for threshold/detection_filter and tighter buffers before you disable it. An alert firing 5,000 times has told you nothing 4,999 times.
  • Version and source everything. rev, created_at/updated_at, and a reference are what let you manage rules at scale; suricata-update expects that discipline when it merges your local rules with feeds.
  • Right-size the action. Ship as alert first, watch it in production, and only promote a high-confidence rule to drop once you trust it — a false-positive drop is an outage.

Testing your rules before you trust them

A rule you have not tested is a hypothesis, not a control. This is where Scapy closes the loop: craft the exact packet your rule should catch, replay it, and confirm the alert.

from scapy.all import *

pkt = IP(dst="10.0.0.53")/UDP(dport=53)/DNS(
        rd=1, qd=DNSQR(qname="a"*45 + ".exfil.example"))
wrpcap("test_tunnel.pcap", pkt)
Enter fullscreen mode Exit fullscreen mode 
suricata -r test_tunnel.pcap -c suricata.yaml -S local.rules -l ./logs/
jq -c 'select(.event_type=="alert").alert.signature' logs/eve.json
Enter fullscreen mode Exit fullscreen mode

One caveat worth internalizing: if you edit a field on a packet Scapy already built, it does not recompute checksums until it re-serializes. Read such a doctored pcap back with tcpdump -v and you will see bad cksum. That is expected — Scapy fixes the checksum on send, not on assignment.

Part 6 — Tying it together

A realistic investigation is not one tool; it is a relay:

  1. Reduce with SiLK — find the handful of flows that are anomalous by volume, timing, or destination.
  2. Reconstruct with Zeek — pull the DNS, HTTP, and TLS logs for those hosts and build a timeline off the shared uid.
  3. Confirm with tcpdump/tshark — carve the exact packets and read the payload where it is not encrypted.
  4. Operationalize with Suricata — encode what you found as a rule so the next occurrence pages you instead of hiding in a pcap.
  5. Verify with Scapy — replay a synthetic version of the attack and prove the rule fires.

Detection engineering is that last step done deliberately. Every incident should leave behind a tested rule, so your coverage compounds instead of resetting with each investigation.

Pitfalls I learned the slow way

  • Match parsed fields, not raw payload, whenever a buffer exists. http.host; content:"x" is precise and fast; a bare content:"x" scans everything and misfires.
  • Scope with flow. Most noisy rules are noisy because they match both directions or unestablished traffic.
  • Threshold high-frequency signals. A rule that fires 5,000 times has told you nothing 4,999 times.
  • Know your offsets. DNS additional records are udp[18:2], not the question count (udp[12:2]). A rule built on the wrong offset looks fine and silently never matches.
  • Test with real bytes. Craft the packet, replay it, read the alert. Trust nothing you have not seen fire.

Further reading

The per-tool cheat sheets that go with this article — tcpdump, tshark, Zeek, Snort, Suricata, SiLK, and Scapy — are in the repository alongside it.

Top comments (0)