Rençber AKMAN

Posted on Aug 1

Module 3: Information Gathering and Vulnerability Scanning

#cybersecurity #informationgathering #penetrationtesting #vulnerabilityassessment

CompTIA PenTest+ / Ethical Hacking Certification Series

Professional Reference Guide — GitHub Edition

Covers: Passive Reconnaissance · OSINT · DNS · Social Media · Cryptographic Analysis · Shodan

3.0 Introduction
3.1 Performing Passive Reconnaissance

3.0 Introduction

Module Overview: Information Gathering and Vulnerability Scanning

Module Objective: Perform information gathering and vulnerability scanning activities at a professional, senior-level standard.

Before a single exploit is launched, before a single payload is crafted, every professional penetration tester invests significant time in a discipline that separates competent practitioners from exceptional ones: information gathering. The reconnaissance phase is the intelligence foundation upon which the entire attack strategy is built. The quality of your reconnaissance directly determines the quality of your attack.

Why This Module is the Most Critical in the Entire Penetration Testing Process

Consider the following reality: a skilled penetration tester with 10 hours of thorough reconnaissance and 2 hours of exploitation will consistently outperform a tester with 1 hour of reconnaissance and 11 hours of exploitation. This is because:

Attack surface knowledge — You cannot attack what you do not know exists
Targeted exploitation — Knowing specific versions, technologies, and configurations enables precise attack selection
Stealth — Passive reconnaissance leaves zero traces in target logs
Social engineering precision — Detailed personnel and organizational intelligence enables highly credible pretexts
Scope awareness — Thorough OSINT reveals assets the client may not even know they have

The Reconnaissance-Attack Continuum

PASSIVE RECON → ACTIVE RECON → SCANNING → ENUMERATION → EXPLOITATION → POST-EXPLOITATION
     (This Module Section 3.1)  (Section 3.2)  (Sections 3.3/3.4)

Passive Reconnaissance is the first and most foundational phase. It involves gathering intelligence about a target using only publicly available information sources, without making any direct contact with the target's systems. The target never knows you are collecting this information.

Active Reconnaissance follows and involves directly interacting with target systems — sending probes, queries, and packets — to enumerate live systems, open ports, and services.

Vulnerability Scanning then uses the intelligence gathered in both recon phases to identify specific security weaknesses in enumerated systems.

Module Topics at a Glance

Section	Topic	Objective
3.1	Performing Passive Reconnaissance	Collect intelligence without touching target systems
3.2	Performing Active Reconnaissance	Directly probe target systems to enumerate infrastructure
3.3	Understanding the Art of Performing Vulnerability Scans	Conduct structured, methodical vulnerability scanning
3.4	Understanding How to Analyze Vulnerability Scan Results	Interpret, prioritize, and act on scan findings

3.1 Performing Passive Reconnaissance

3.1.1 Overview

Passive reconnaissance is the discipline of collecting as much intelligence as possible about a target organization using only publicly available information — information that exists in the open and can be accessed without the target's knowledge.

The term "passive" is critical: during this phase, you generate zero network traffic to the target. No pings. No port scans. No HTTP requests to the target's web server. Every data point is gathered from third-party sources, public databases, archived data, and open web resources.

Why Passive Reconnaissance Matters at the Senior Level

Junior penetration testers often treat reconnaissance as a checklist to complete before getting to the "real work" of exploitation. Senior penetration testers understand that reconnaissance is the work. The penetration testing firm Offensive Security, authors of Kali Linux and creators of the OSCP certification, teach that a penetration tester should spend at least 30-40% of total engagement time on reconnaissance.

The professional reasons:

1. Legal protection: Passive recon is never illegal. Accessing public information carries zero criminal risk, while premature active scanning of the wrong IP address is a CFAA violation.

2. Attack precision: Knowing that a target runs Apache Tomcat 9.0.41 on a specific IP enables you to immediately cross-reference known CVEs. Without this knowledge, you are scanning blindly.

3. Organizational intelligence: Passive recon reveals the human attack surface — executives, IT staff, vendors, technologies in use — enabling social engineering attacks that technical controls cannot stop.

4. Shadow IT discovery: Passive recon routinely reveals subdomains, applications, and cloud assets that the client's IT team does not know exist. These unmanaged assets are frequently the easiest entry points.

5. Timeline intelligence: Web archives reveal what technologies a target used in the past, sometimes exposing legacy systems still in use.

3.1.2 Active Reconnaissance vs. Passive Reconnaissance

Understanding the precise boundary between passive and active reconnaissance is both a technical and a legal necessity.

Passive Reconnaissance — Defined

Passive reconnaissance collects information from sources that are:

Publicly indexed and accessible to anyone
Third-party resources (not the target's own infrastructure)
Archived or cached versions of target information
Volunteered information (press releases, job listings, social media)

The defining test: "Did my action generate any network traffic or log entries on the target's systems?" If no — it is passive.

Examples of passive reconnaissance activities:

Looking up DNS records using a third-party resolver
Searching LinkedIn for employees of the target organization
Examining cached versions of the target's website via Google Cache or Wayback Machine
Reading the target's press releases and annual reports
Searching Shodan for the target's IP ranges
Examining SSL certificate transparency logs
Running WHOIS lookups through a third-party service
Searching GitHub for code related to the target organization
Examining job listings to identify technologies in use
Analyzing file metadata from documents published on the target's website

Active Reconnaissance — Defined

Active reconnaissance involves directly interacting with the target's systems and infrastructure.

The defining test: "Does my action generate network traffic that the target could log, detect, or block?" If yes — it is active.

Examples of active reconnaissance activities:

Port scanning the target's IP addresses (nmap)
Banner grabbing from the target's servers
Sending HTTP requests to the target's web application
Tracerouting to the target's infrastructure
Performing DNS zone transfer attempts against the target's DNS servers
Crawling the target's website
Sending ping probes to the target's IP ranges

The Legal Distinction

This distinction has direct legal implications:

Aspect	Passive Recon	Active Recon
Legal risk	None — accessing public info	Potential CFAA violation if unauthorized
Detection risk	Zero — no target interaction	High — generates logs on target systems
Pre-authorization	Can be performed before authorization is signed	Must only be performed after authorization
Log evidence	No traces on target	Target's IDS/IPS/firewall logs activity
Timing	Can begin day 1 of engagement	Begins only after ROE is signed

Critical professional practice: Many penetration testers begin passive reconnaissance immediately after being engaged — even before the contract is finalized — because it generates zero legal risk and the intelligence gathered informs the scoping conversation with the client.

The Passive-Active Spectrum

Some activities exist in a gray zone:

Activity	Classification	Why
Google search for target domain	Passive	No target interaction
Accessing target's public website	Active	Generates server logs on target
WHOIS via third-party service	Passive	Third party handles the lookup
DNS query via your own resolver	Active	Your resolver queries target's authoritative DNS
DNS query via a third-party tool	Passive	Third party generates the query
Shodan search for target IPs	Passive	Shodan already scanned; you view results
Running nmap against target IP	Active	Direct packets to target
Certificate Transparency logs	Passive	Accessing third-party CT log databases

Understanding this spectrum allows you to plan exactly what can be done before authorization and what requires a signed contract.

3.1.3 The OSINT Methodology — How Professionals Think

OSINT is not random searching. Professional intelligence analysts follow a structured methodology derived from military and intelligence community practices.

The Intelligence Cycle

┌─────────────────────────────────────────────────────────┐
│                   INTELLIGENCE CYCLE                     │
│                                                          │
│  1. PLANNING & DIRECTION                                 │
│     Define intelligence requirements                     │
│     What do we need to know? Why?                       │
│          ↓                                               │
│  2. COLLECTION                                           │
│     Gather raw data from multiple sources               │
│          ↓                                               │
│  3. PROCESSING                                           │
│     Convert raw data into usable format                 │
│          ↓                                               │
│  4. ANALYSIS & PRODUCTION                               │
│     Evaluate, correlate, and interpret data             │
│          ↓                                               │
│  5. DISSEMINATION                                        │
│     Deliver intelligence to decision makers             │
│          ↓                                               │
│  6. FEEDBACK                                            │
│     Refine requirements based on results                │
│          └────────────────────────────────────┘         │
└─────────────────────────────────────────────────────────┘

OSINT Intelligence Requirements for Penetration Testing

At the start of passive recon, define what you need to know:

Infrastructure Intelligence:

What IP address ranges does the organization own?
What domain names and subdomains does the organization operate?
What hosting providers and cloud services does the organization use?
What technologies (web servers, frameworks, databases) are in use?
What externally accessible services are running?

Organizational Intelligence:

Who are the key technical personnel (IT staff, developers, security team)?
What does the organizational chart look like?
What vendors and third-party services does the organization use?
What business units exist and what are their functions?

Human Intelligence (HUMINT):

What are the email address formats used by the organization?
What information do employees publicly share about their work and technologies?
What skills do employees list (revealing technologies in use)?
What security awareness level do employees demonstrate?

Reputational Intelligence:

Has the organization suffered previous data breaches?
What is the organization's public security posture?
Are there leaked credentials in breach databases?
What does the organization's dark web footprint look like?

Historical Intelligence:

What did the organization's infrastructure look like in the past?
What technologies have they used and potentially still use?
What security incidents have been publicly reported?

The Pivot Model

Professional OSINT analysts use a technique called pivoting — using one piece of intelligence to unlock additional intelligence. Every data point discovered can be used to find more:

Organization Name
    ├── Domain Name
    │       ├── IP Addresses (DNS A records)
    │       │       ├── ASN / IP Range
    │       │       │       └── Other IPs in the range → More targets
    │       │       └── Geolocation → Data center / hosting provider
    │       ├── Subdomains
    │       │       └── Each subdomain → New IP → New services
    │       ├── Mail Servers (MX records)
    │       │       └── Email provider → Office 365? G Suite?
    │       └── SSL Certificates
    │               └── Subject Alternative Names → Hidden subdomains
    ├── Executive Names (from LinkedIn)
    │       ├── Email address (using email format)
    │       │       └── Breach data → Leaked passwords → Credential stuffing
    │       └── Social media → Technology disclosures
    └── Job Listings
            └── Technology stack ("requires experience with AWS, Kubernetes, HashiCorp Vault")

This pivot model means that starting with only an organization's name, a skilled analyst can map out the entire attack surface before touching a single target system.

3.1.4 OSINT Tools — The Complete Professional Arsenal

OSINT Framework

Website: https://osintframework.com/

Type: Reference/Navigation Tool

Cost: Free

OSINT Framework is the foundational resource for any OSINT practitioner. Created by Justin Nordine, it is an interactive, hierarchically organized map of hundreds of OSINT tools and techniques, organized by the type of data you have (starting point) and what you want to find.

Structure of OSINT Framework:

The framework is organized as a tree starting from the type of indicator you possess:

OSINT Framework Root
├── Username
│   ├── Username Search Engines (Sherlock, WhatsMyName)
│   ├── Social Networks
│   └── Forums / Communities
├── Email Address
│   ├── Email Reputation
│   ├── Breach Data
│   └── Associated Accounts
├── Domain Name
│   ├── WHOIS Records
│   ├── DNS Records
│   ├── Subdomains
│   └── Website Analysis
├── IP Address
│   ├── Geolocation
│   ├── Reverse DNS
│   └── Network Information
├── Image / Photo
│   ├── Reverse Image Search
│   └── Facial Recognition
├── Phone Number
│   ├── Carrier Lookup
│   └── Social Media Linked
└── ... (hundreds more branches)

How professionals use OSINT Framework:

Start with what you have (e.g., an email address from a job listing)
Navigate to that branch in the framework
Identify which tools are most appropriate for your objective
Execute the tools in sequence, pivoting from each result

Notation in OSINT Framework:

(T) — Tool (requires installation or technical setup)
(D) — Dynamic (content changes based on input)
(R) — Requires registration/account
(M) — Malware warning (use with caution)

OSINT Combine

Website: https://www.osintcombine.com/

Type: Tool collection and automation platform

Cost: Partially free; premium features available

OSINT Combine is a professional-grade platform developed by Australian OSINT specialists containing tools designed for specific, high-value OSINT tasks. It differentiates itself by focusing on automation and efficiency — reducing the manual effort of common OSINT workflows.

Key tools available on OSINT Combine:

Image Metadata Viewer: Extracts and displays EXIF metadata from images (GPS coordinates, camera model, timestamps) directly in the browser without downloading.

Social Media Search: Cross-platform username and content searches designed to overcome the limitations of native platform search interfaces.

Map Searching Tools: Specialized tools for geolocation analysis and map-based OSINT (verifying locations from photos, tracking movements from social media content).

Domain Investigation Tools: DNS history, WHOIS lookups, and subdomain enumeration integrated into a unified workflow.

Bulk Data Processing: Tools for processing large datasets (e.g., processing multiple usernames or email addresses simultaneously).

Professional Application:

OSINT Combine is particularly valuable for:

Social media investigations (identifying accounts across platforms)
Geolocation of images shared by employees (potentially revealing office locations, travel patterns, physical security details)
Bulk processing when handling large employee lists from LinkedIn

SMART — Start.me Aggregated Resource Tool

Website: https://smart.myosint.training/

Type: OSINT bookmark aggregator and search interface

Cost: Free

Created by: My OSINT Training (MOT) Team

SMART solves a specific problem in OSINT practice: the fragmentation of resources. Thousands of practitioners maintain OSINT resource lists on the start.me bookmarking platform. SMART indexes all of these public lists and provides a unified search interface across them.

What makes SMART valuable:

When you are looking for tools related to a specific intelligence requirement — say, "maritime vessel tracking" or "corporate registration databases for Brazil" — SMART quickly surfaces specialized resources that would otherwise require extensive searching.

Use cases:

Finding country-specific OSINT resources (corporate registries, electoral rolls, court records)
Discovering niche tools for specific data types
Building a comprehensive resource list for a specific engagement type

SpiderFoot

Website: https://www.spiderfoot.net/

GitHub: https://github.com/smicallef/spiderfoot

Type: Automated OSINT reconnaissance platform

Cost: Open-source (SpiderFoot HX cloud version has costs)

Installation: pip3 install spiderfoot or from GitHub

SpiderFoot is one of the most powerful automated OSINT tools in existence. It takes a single target indicator (IP address, domain name, email address, person name, or ASN) and automatically queries over 200 data sources to build a comprehensive intelligence profile.

How SpiderFoot Works:

SpiderFoot operates on a modular architecture. Each module queries a specific data source. When one module returns data, it triggers other modules that can use that data as input — creating an automated pivot chain.

Input: targetcompany.com
    ↓
DNS Module → IP addresses: 203.0.113.50, 203.0.113.51
    ↓
IP Whois Module → ASN: AS12345, Organization: Target Company Inc.
    ↓
Netblock Module → Full IP range: 203.0.113.0/24
    ↓
Port Scanner Module → Open ports on all IPs in range
    ↓
Banner Grab Module → Service banners and versions
    ↓
Certificate Module → SSL certs → Subject Alternative Names → New subdomains
    ↓
Shodan Module → Shodan data for each IP
    ↓
Breach Module → Check emails against HaveIBeenPwned
    ... (200+ modules)

SpiderFoot Modules of Highest Value for Penetration Testers:

Module	Data Source	Intelligence Gathered
`sfp_dnsresolve`	DNS	IP addresses for domains
`sfp_ssl`	SSL certificate analysis	Subject alternative names, cert history
`sfp_shodan`	Shodan API	Open ports, services, banners
`sfp_whois`	WHOIS	Registrant info, nameservers
`sfp_hunter`	Hunter.io	Email addresses from domain
`sfp_hibp`	HaveIBeenPwned	Breach exposure of emails
`sfp_linkedin`	LinkedIn	Employee names and roles
`sfp_github`	GitHub	Source code, credentials, configs
`sfp_pastebin`	Pastebin	Leaked data mentioning target
`sfp_virustotal`	VirusTotal	URL/IP reputation, malware association
`sfp_threatcrowd`	ThreatCrowd	Threat intelligence correlation
`sfp_googlesearch`	Google	Indexed pages, exposed files

Running SpiderFoot:

# Install
pip3 install spiderfoot

# Launch web interface
spiderfoot -l 127.0.0.1:5001

# Command line scan
spiderfoot -s targetcompany.com -t INTERNET_NAME -o json -q

# Scan with specific modules only
spiderfoot -s targetcompany.com -m sfp_dns,sfp_ssl,sfp_shodan -o json

# Full passive scan (no active modules)
spiderfoot -s targetcompany.com --type passive -o json

SpiderFoot vs. Manual OSINT:

Approach	Time for Full Domain Recon	Breadth
Manual OSINT	4-8 hours	Depends on analyst skill
SpiderFoot automated	15-45 minutes	200+ sources automatically

SpiderFoot does not replace human analysis — it accelerates data collection so the analyst can focus on interpretation and pivoting.

Recon-ng

GitHub: https://github.com/lanmaster53/recon-ng

Documentation: https://github.com/lanmaster53/recon-ng/wiki

Type: Modular web reconnaissance framework

Cost: Free / Open-source

Platform: Linux (included in Kali Linux)

Created by: Tim Tomes (LaNMaSteR53)

Recon-ng is a full-featured web reconnaissance framework written in Python. Its design is deliberately modeled after Metasploit — experienced penetration testers who know Metasploit will find recon-ng immediately familiar. It provides a powerful, module-based CLI for conducting systematic OSINT campaigns.

Why recon-ng is a professional standard:

Database backend: All results are stored in a local SQLite database, enabling complex queries, cross-referencing, and persistent storage across sessions
Workspace isolation: Create separate workspaces for each client engagement, keeping data cleanly separated
Module ecosystem: Hundreds of modules covering every aspect of OSINT
API key management: Centralized management of API keys for services like Shodan, VirusTotal, Hunter.io
Reporting: Built-in report generation from collected data
Automation: Scripting capability for repeatable reconnaissance workflows

recon-ng Core Concepts:

Workspaces: Isolated databases for each engagement

[recon-ng][default] > workspaces create client_target_co
[recon-ng][client_target_co] >

Modules: The intelligence-gathering engines

[recon-ng][client_target_co] > modules search domains
[*] Searching for 'domains'...

  Discovery
  ---------
    discovery/info_disclosure/interesting_files

  Recon
  -----
    recon/domains-contacts/hunter_io
    recon/domains-credentials/pwnedlist_domain_credentials
    recon/domains-domains/brute_suffix
    recon/domains-hosts/bing_domain_web
    recon/domains-hosts/brute_hosts
    recon/domains-hosts/certificate_transparency
    recon/domains-hosts/google_site_web
    recon/domains-hosts/netcraft
    recon/domains-hosts/shodan_hostname
    recon/domains-hosts/ssl_san
    recon/domains-vulnerabilities/punkspider
    recon/domains-vulnerabilities/xssed
    recon/domains-vulnerabilities/xssposed

The recon-ng Module Naming Convention:

category/subcategory/source

recon/domains-hosts/certificate_transparency
 │         │              │
 │         │              └── Data source used
 │         └── What you HAVE → What you GET (domains → hosts)
 └── Module category

Common recon-ng Workflow:

# Launch recon-ng
recon-ng

# Create workspace for engagement
workspaces create targetco_engagement_2024

# Add seed domain
db insert domains targetco.com

# Find subdomains via certificate transparency
modules load recon/domains-hosts/certificate_transparency
run

# Find subdomains via Bing
modules load recon/domains-hosts/bing_domain_web
run

# Resolve all discovered hosts to IPs
modules load recon/hosts-hosts/resolve
run

# Find email addresses for domain
modules load recon/domains-contacts/hunter_io
options set SOURCE targetco.com
run

# Check emails against breach data
modules load recon/contacts-credentials/hibp_breach
run

# Generate HTML report
modules load reporting/html
options set FILENAME /tmp/targetco_recon_report.html
run

High-Value recon-ng Modules:

Module	Input	Output
`recon/domains-hosts/certificate_transparency`	Domain	Subdomains from CT logs
`recon/domains-hosts/shodan_hostname`	Domain	IPs and ports from Shodan
`recon/domains-hosts/brute_hosts`	Domain	Subdomain brute force
`recon/domains-contacts/hunter_io`	Domain	Email addresses
`recon/contacts-credentials/hibp_breach`	Email	Breach data
`recon/hosts-ports/shodan_ip`	IP Address	Open ports from Shodan
`recon/netblocks-companies/whois_orgs`	Netblock	Organization info
`recon/companies-multi/whois_miner`	Company	All WHOIS data
`recon/profiles-profiles/linkedin_auth`	LinkedIn URL	Profile details

recon-ng API Key Setup:

recon-ng requires API keys for many of its most powerful modules:

keys add shodan_api [YOUR_KEY]
keys add hunter_api [YOUR_KEY]
keys add virustotal_api [YOUR_KEY]
keys add censys_id [YOUR_ID]
keys add censys_secret [YOUR_SECRET]

Maltego

Website: https://www.maltego.com/

Type: Visual intelligence and link analysis platform

Cost: Community (free, limited), Pro ($999/year), Enterprise

Platform: Cross-platform (Windows, macOS, Linux)

Maltego is the industry-standard tool for visual OSINT analysis and link analysis. Unlike CLI tools, Maltego presents intelligence graphically — as a network graph showing relationships between entities (people, organizations, domains, IPs, emails, social profiles).

Why Maltego is used at the enterprise level:

Visual relationship mapping: Instantly reveals connections between entities that would take hours to identify in text-based tools
Transform-based automation: "Transforms" are automated queries that expand the graph with new intelligence
Maltego Transform Hub: Marketplace of transforms from commercial data providers (Shodan, Have I Been Pwned, VirusTotal, etc.)
Collaboration: Teams can share investigation graphs
Evidence preservation: The graph is a defensible record of the investigation methodology

Core Maltego Entity Types:

Person, EmailAddress, PhoneNumber, Organization
Domain, URL, Website, DNSName, NSRecord, MXRecord
IPv4Address, Netblock, ASNumber
Social media profiles (Twitter, LinkedIn, Facebook)
File, Document, Phrase

Professional Maltego Workflow:

Add seed entity (e.g., targetco.com as a Domain entity)
Run DNS transforms → discovers A, MX, NS records + IP addresses
Run WHOIS transforms → discovers registrant info
Run SSL transforms → discovers Subject Alternative Names (new subdomains)
Run email transforms → discovers email addresses (via Hunter.io, etc.)
Run breach transforms → checks emails against HaveIBeenPwned
Run social media transforms → finds LinkedIn/Twitter profiles
Run Shodan transforms → enriches IP data with port/service information

The result is a comprehensive, visual map of the target's entire digital presence and the relationships between all discovered entities.

theHarvester

GitHub: https://github.com/laramies/theHarvester

Type: Email and subdomain harvester

Cost: Free / Open-source

Platform: Linux (included in Kali Linux)

theHarvester is one of the oldest and most reliable passive reconnaissance tools. Its specific focus is gathering email addresses, subdomains, and employee names from multiple public data sources.

Data Sources Supported:

Source	What it Finds
Google	Emails, subdomains, hosts
Bing	Emails, subdomains, hosts
DuckDuckGo	Emails, hosts
LinkedIn	Employee names
Twitter	Usernames, emails
Shodan	Hosts, open ports
CertSpotter	Subdomains via CT logs
DNSdumpster	DNS info, subdomains
Netcraft	Subdomains
VirusTotal	Subdomains
Hunter.io	Emails
Intelx	Emails, hosts (requires API key)

Usage:

# Basic domain email and subdomain harvest
theHarvester -d targetco.com -b all

# Specific data sources
theHarvester -d targetco.com -b google,linkedin,shodan

# Limit results and save to file
theHarvester -d targetco.com -b all -l 500 -f /tmp/harvest_results.html

# Include subdomains in search
theHarvester -d targetco.com -b all -s

# Specify virtual host verification
theHarvester -d targetco.com -b all -v

What theHarvester Discovers:

[*] Emails found: 23
--------------------------------------------------
j.smith@targetco.com
a.johnson@targetco.com
m.williams@targetco.com
security@targetco.com
admin@targetco.com

[*] Hosts found: 47
--------------------------------------------------
mail.targetco.com:203.0.113.10
vpn.targetco.com:203.0.113.15
dev.targetco.com:10.0.1.100  ← Shadow IT discovery
staging.targetco.com:203.0.113.20  ← Pre-production environment

The discovery of dev.targetco.com and staging.targetco.com — development and staging servers — is one of the most valuable passive recon findings. These environments frequently have weaker security controls than production systems.

Sherlock

GitHub: https://github.com/sherlock-project/sherlock

Type: Username cross-platform search tool

Cost: Free / Open-source

Sherlock searches for a specific username across 300+ social media platforms and websites simultaneously. This is invaluable for:

Finding all public profiles associated with an employee's known username
Identifying personal accounts that may disclose sensitive information
Building a complete social profile of a target individual

# Install
pip3 install sherlock-project

# Search for username
sherlock johndoe_security

# Multiple usernames
sherlock johndoe johndoe_security j.doe

# Specify output file
sherlock johndoe --output /tmp/johndoe_results.txt

hacker.org

Website: https://hacker.org/

Type: Cybersecurity skill development platform / CTF-style challenges

Cost: Free (registration required for challenges)

hacker.org is a training and practice platform designed for developing offensive security skills through hands-on challenges. It is categorized as a skill development resource rather than an OSINT data source, but it is relevant to the information gathering module because it develops the analytical and technical thinking required for reconnaissance.

What hacker.org offers:

Programming challenges: Logic and algorithm problems requiring code solutions
Cryptography challenges: Breaking and implementing cryptographic systems
Web security challenges: Finding and exploiting web vulnerabilities in safe, legal environments
Network analysis challenges: Analyzing packet captures and network protocols
Steganography challenges: Finding hidden data in images and files
CTF-style progressions: Increasing difficulty levels that build systematically on each other

How it builds reconnaissance skills:

Many reconnaissance techniques require exactly the analytical skills developed on hacker.org:

Identifying hidden information in files and images (steganography → file metadata analysis)
Understanding cryptographic weaknesses (crypto challenges → SSL/TLS vulnerability identification)
Finding information in unexpected places (web challenges → advanced Google dorking)
Scripting and automation (programming challenges → automated OSINT tool development)

Professional positioning: hacker.org and similar platforms (Hack The Box, TryHackMe, PicoCTF) are increasingly referenced in job interviews as evidence of hands-on skill development.

3.1.5 DNS Lookups — Deep Dive

The Domain Name System (DNS) is one of the most information-rich sources available during passive reconnaissance. DNS records publicly document an organization's infrastructure in ways most organizations do not fully appreciate.

DNS Fundamentals — What Every Senior Penetration Tester Must Know

DNS translates human-readable domain names into machine-readable IP addresses. But its function extends far beyond simple name resolution — it is a distributed database containing multiple record types that reveal extensive infrastructure intelligence.

DNS Architecture:

Client Query: "What is the IP of www.targetco.com?"

Resolver (Recursive DNS Server)
    ↓ (queries if not cached)
Root Name Server → "Ask .com TLD servers"
    ↓
.com TLD Server → "Ask targetco.com's authoritative name servers"
    ↓
Authoritative NS for targetco.com → "203.0.113.50"
    ↓
Answer returned to client: 203.0.113.50

Intelligence value at each layer:

Root server queries: Reveal what TLD the organization uses (.com, .gov, .mil, .co.uk — indicates jurisdiction)
Authoritative name servers: Reveal DNS hosting provider (AWS Route 53, Cloudflare, GoDaddy — intelligence about infrastructure providers)
DNS record types: Each type reveals specific infrastructure details

Complete DNS Record Type Reference

A Record (Address Record)

Maps a hostname to an IPv4 address. The most fundamental DNS record.

# Query A record
dig targetco.com A
nslookup targetco.com

# Response
targetco.com.    300    IN    A    203.0.113.50

Intelligence value:

The IP address enables further intelligence gathering (Shodan, WHOIS for the IP, geolocation)
The TTL (Time To Live, in seconds — here: 300 seconds = 5 minutes) reveals caching behavior; very low TTLs suggest load balancing or CDN use; very high TTLs suggest static infrastructure
Multiple A records for the same hostname indicate load balancing or CDN

AAAA Record (IPv6 Address Record)

Maps a hostname to an IPv6 address. Organizations increasingly deploy IPv6, and IPv6 addresses are often less well-monitored than IPv4.

dig targetco.com AAAA

Intelligence value:

IPv6 often reveals direct IP addresses even when IPv4 is behind a CDN (like Cloudflare)
IPv6 address blocks often reveal the organization's ISP or hosting provider
Organizations frequently apply less rigorous security controls to IPv6 paths

MX Record (Mail Exchanger Record)

Specifies the mail servers responsible for accepting email for a domain.

dig targetco.com MX

# Response
targetco.com.    3600    IN    MX    10    mail1.targetco.com.
targetco.com.    3600    IN    MX    20    mail2.targetco.com.
targetco.com.    3600    IN    MX    30    aspmx.l.google.com.   ← G Suite!

Intelligence value:

Reveals email provider (Google Workspace, Microsoft 365, self-hosted Exchange)
Google Workspace indicators: aspmx.l.google.com, alt1.aspmx.l.google.com
Microsoft 365 indicators: targetco-com.mail.protection.outlook.com
Self-hosted Exchange: Typically a subdomain like mail.targetco.com pointing to a corporate IP
Email platform reveals authentication methods, phishing opportunities, and password spray targets
Backup MX records (lower priority, higher number) sometimes point to less-secure mail relay servers

NS Record (Name Server Record)

Identifies the authoritative DNS servers for a domain.

dig targetco.com NS

# Response
targetco.com.    86400    IN    NS    ns1.targetco.com.
targetco.com.    86400    IN    NS    ns2.targetco.com.

# OR (revealing DNS hosting provider)
targetco.com.    86400    IN    NS    ns-1234.awsdns-12.com.     ← AWS Route 53
targetco.com.    86400    IN    NS    ns-5678.awsdns-34.co.uk.

Intelligence value:

Reveals DNS hosting provider (AWS Route 53, Cloudflare, Azure DNS, Google Cloud DNS)
Self-hosted NS records (ns1.targetco.com) reveal additional IP addresses to investigate
DNS provider may have security implications (DNS hijacking attacks target specific providers)
Cloudflare NS records often mean IPv4 addresses are proxied/hidden

SOA Record (Start of Authority)

Contains administrative information about a DNS zone, including the primary name server and the email address of the zone administrator.

dig targetco.com SOA

# Response
targetco.com.    3600    IN    SOA    ns1.targetco.com. dnsadmin.targetco.com. (
                                      2024010101  ; Serial
                                      3600        ; Refresh
                                      900         ; Retry
                                      604800      ; Expire
                                      300 )       ; Minimum TTL

Intelligence value:

dnsadmin.targetco.com — The second field is the DNS administrator's email address (replace the first . with @): dnsadmin@targetco.com
This is a direct email address for a technical administrator
Serial number format often reveals the last update date (common format: YYYYMMDDNN)

TXT Record (Text Record)

A versatile record type containing arbitrary text. Used for numerous verification and configuration purposes.

dig targetco.com TXT

# Common responses
targetco.com.    3600    IN    TXT    "v=spf1 include:_spf.google.com include:sendgrid.net ip4:203.0.113.50 ~all"
targetco.com.    3600    IN    TXT    "MS=ms12345678"
targetco.com.    3600    IN    TXT    "google-site-verification=AbCdEfGhIjKlMnOpQrStUvWxYz"
targetco.com.    3600    IN    TXT    "atlassian-domain-verification=AbCdEf12345"

Intelligence value — TXT records are a goldmine:

TXT Record Content	Intelligence Revealed
`v=spf1 include:_spf.google.com`	Uses Google Workspace for email
`v=spf1 include:sendgrid.net`	Uses SendGrid for marketing emails
`v=spf1 include:_spf.salesforce.com`	Uses Salesforce (email integration)
`v=spf1 ip4:203.0.113.50`	Mail server IP (direct IP revelation)
`MS=ms12345678`	Microsoft 365 domain verification — organization is on Microsoft 365
`google-site-verification=...`	Google Search Console verification — uses Google services
`atlassian-domain-verification=...`	Uses Atlassian products (Jira, Confluence, Bitbucket)
`docusign=...`	Uses DocuSign for e-signatures
`stripe-verification=...`	Uses Stripe for payments (financial data!)
`facebook-domain-verification=...`	Facebook/Meta advertising integration
`_dmarc` (separate record)	DMARC policy (reveals email security posture)

DMARC Record:

dig _dmarc.targetco.com TXT

# Response options
"v=DMARC1; p=none; rua=mailto:dmarc@targetco.com"    ← No enforcement (easy phishing)
"v=DMARC1; p=quarantine; ..."                          ← Moderate protection
"v=DMARC1; p=reject; ..."                              ← Strong protection

# p=none means phishing emails spoofing @targetco.com will be DELIVERED

DMARC intelligence:

p=none — The organization does not enforce DMARC. Spoofed emails using their domain will be delivered to recipients. Directly relevant to phishing pre-text design.
The rua (reporting URI) reveals an email address for DMARC reports — a real internal email address.

SPF Record Analysis:

dig targetco.com TXT | grep spf

# v=spf1 include:_spf.google.com include:sendgrid.net include:_spf.salesforce.com ip4:203.0.113.0/24 ip6:2001:db8::/32 ~all

This single SPF record reveals:

Google Workspace (corporate email)
SendGrid (marketing/transactional email — phishing campaigns often come from here)
Salesforce (CRM platform)
Direct IP range 203.0.113.0/24 (mail server IPs)
IPv6 range 2001:db8::/32

CNAME Record (Canonical Name Record)

Maps one hostname to another hostname (an alias).

dig www.targetco.com CNAME

# Responses
www.targetco.com.    300    IN    CNAME    targetco.com.
www.targetco.com.    300    IN    CNAME    d1234567890.cloudfront.net.    ← CloudFront CDN
www.targetco.com.    300    IN    CNAME    targetco.azurewebsites.net.    ← Azure App Service
www.targetco.com.    300    IN    CNAME    targetco.github.io.            ← GitHub Pages

Intelligence value:

Reveals content delivery networks (CloudFront, Cloudflare, Fastly, Akamai)
Reveals cloud hosting services (Azure Web Apps, AWS Elastic Beanstalk, Heroku, GitHub Pages)
CNAMEs to third-party services reveal vendor relationships
Subdomain takeover: If a CNAME points to a third-party service that is no longer configured, the subdomain may be vulnerable to takeover

PTR Record (Pointer Record / Reverse DNS)

Maps an IP address back to a hostname. The reverse of an A record.

# Reverse DNS lookup
dig -x 203.0.113.50
nslookup 203.0.113.50

# Response
50.113.0.203.in-addr.arpa.    3600    IN    PTR    mail.targetco.com.

Intelligence value:

Reveals the hostname associated with an IP address
Particularly valuable when you have an IP from a log, packet capture, or other source and need to identify the system
Reveals infrastructure naming conventions (the pattern used in mail.targetco.com vs web01.prod.targetco.com reveals a lot about internal structure)

SRV Record (Service Record)

Specifies the location (hostname and port) of servers for specific services.

dig _sip._tcp.targetco.com SRV
dig _autodiscover._tcp.targetco.com SRV
dig _xmpp-server._tcp.targetco.com SRV

# Response
_autodiscover._tcp.targetco.com.    3600    IN    SRV    0 0 443 autodiscover.targetco.com.

Intelligence value:

_autodiscover._tcp → Microsoft Exchange/Office 365 autodiscovery — confirms Microsoft 365 and reveals Exchange server
_sip._tcp → SIP/VoIP server → Voice over IP infrastructure
_xmpp-server._tcp → XMPP/Jabber server → Instant messaging
_kerberos._tcp → Kerberos → Active Directory present (major finding)
_ldap._tcp → LDAP → Active Directory present

CAA Record (Certification Authority Authorization)

Specifies which Certificate Authorities (CAs) are authorized to issue SSL/TLS certificates for the domain.

dig targetco.com CAA

# Response
targetco.com.    3600    IN    CAA    0 issue "letsencrypt.org"
targetco.com.    3600    IN    CAA    0 issue "digicert.com"
targetco.com.    3600    IN    CAA    0 issuewild "digicert.com"

Intelligence value:

Reveals which CAs the organization uses → cert monitoring strategy
If Let's Encrypt only → likely smaller/budget-conscious operations
If DigiCert/Sectigo/Entrust → enterprise-grade certificates
Absence of CAA records means any CA can issue certificates → higher phishing certificate risk

3.1.6 DNS Reconnaissance — Advanced Techniques

Zone Transfer (AXFR)

A DNS zone transfer is the mechanism by which DNS servers replicate zone data to secondary servers. If misconfigured, a zone transfer can be requested by anyone, returning the complete list of all DNS records in the zone.

Attempting a zone transfer:

# Identify nameservers first
dig targetco.com NS

# Attempt zone transfer from each nameserver
dig axfr targetco.com @ns1.targetco.com
dig axfr targetco.com @ns2.targetco.com

# Using nslookup
nslookup
> server ns1.targetco.com
> ls -d targetco.com

Successful zone transfer result:

; <<>> DiG 9.18.0 <<>> axfr targetco.com @ns1.targetco.com
targetco.com.        86400    IN    SOA    ns1.targetco.com. admin.targetco.com. ...
targetco.com.        86400    IN    NS     ns1.targetco.com.
targetco.com.        86400    IN    A      203.0.113.50
www.targetco.com.    86400    IN    A      203.0.113.50
mail.targetco.com.   86400    IN    A      203.0.113.10
vpn.targetco.com.    86400    IN    A      203.0.113.15
dev.targetco.com.    86400    IN    A      10.0.1.100
staging.targetco.com. 86400  IN    A      203.0.113.20
db01.targetco.com.   86400    IN    A      10.0.1.200
internal.targetco.com. 86400 IN    A      10.0.0.1

This instantly reveals the entire internal network structure. Zone transfers are considered active reconnaissance (you're querying the target's DNS server), but the information returned is public (just poorly protected).

Note: Most modern, properly configured DNS servers restrict zone transfers to specific secondary server IPs. Zone transfer success is itself a critical vulnerability finding.

DNS Brute Forcing

When zone transfers fail (as they usually should), subdomain discovery is accomplished by brute forcing — querying the target's DNS server with a list of common subdomain names.

# Using dnsx
dnsx -d targetco.com -w /usr/share/wordlists/seclists/Discovery/DNS/subdomains-top1million-5000.txt

# Using ffuf for DNS brute forcing
ffuf -w /usr/share/wordlists/seclists/Discovery/DNS/subdomains-top1million-5000.txt -u http://FUZZ.targetco.com -v

# Using gobuster for DNS enumeration
gobuster dns -d targetco.com -w /usr/share/wordlists/seclists/Discovery/DNS/subdomains-top1million-110000.txt

# Using amass (passive mode — no active queries)
amass enum -passive -d targetco.com

# Using amass (active mode)
amass enum -active -d targetco.com -brute

Common Subdomain Wordlists:

SecLists (/usr/share/seclists/Discovery/DNS/) — The de-facto standard
subdomains-top1million-5000.txt — Fast, covers most common subdomains
subdomains-top1million-110000.txt — Comprehensive
dns-Jhaddix.txt — Curated by renowned bug bounty hunter Jason Haddix

DNS History and Passive DNS

Historical DNS records reveal:

Previous IP addresses (before migration to cloud/CDN)
Past subdomain configurations
Infrastructure changes over time

Tools for DNS history:

Tool	URL	What it Shows
SecurityTrails	https://securitytrails.com	Complete DNS history, subdomains, IP history
DNSHistory.io	https://dnshistory.org	Historical DNS records
ViewDNS.info	https://viewdns.info	DNS history, reverse IP, IP history
PassiveDNS (Farsight DNSDB)	https://www.farsightsecurity.com	Passive DNS database (enterprise)
RiskIQ Community	https://community.riskiq.com	Passive DNS, certificate history

Why DNS history matters:

Organizations that move from direct hosting to CDN (e.g., Cloudflare) often have their real IP address in DNS history before the move. The CDN hides the real IP in current DNS, but historical records expose it:

Current A record: targetco.com → 104.21.x.x (Cloudflare IP — not the real server)
Historical DNS:   targetco.com → 203.0.113.50 (Real origin server, now bypasses CDN)

Direct access to the origin IP bypasses:

Web Application Firewall (WAF)
DDoS protection
Rate limiting
Bot detection

3.1.7 Identification of Technical and Administrative Contacts

Technical and administrative contacts are directly relevant to penetration testing because:

They are the people who manage the systems being tested
Their contact information enables social engineering pretext construction
Their email addresses are high-value targets for credential phishing
Their technical roles revealed through public profiles expose technologies in use

WHOIS Contact Records

WHOIS records for domain registrations contain:

Registrant contact: The entity (person or organization) that owns the domain
Administrative contact: The person responsible for administrative matters
Technical contact: The person responsible for technical management

whois targetco.com

# Output (pre-GDPR, or for non-privacy-protected registrations)
Domain Name: TARGETCO.COM
Registry Domain ID: 1234567890_DOMAIN_COM-VRSN
Registrar: GoDaddy.com, LLC

Registrant Name: John Smith
Registrant Organization: Target Company Inc.
Registrant Street: 123 Corporate Drive
Registrant City: San Francisco
Registrant State/Province: CA
Registrant Postal Code: 94105
Registrant Country: US
Registrant Phone: +1.4155551234
Registrant Email: john.smith@targetco.com

Admin Name: Jane Doe
Admin Email: it-admin@targetco.com

Tech Name: Bob Johnson
Tech Email: dns-admin@targetco.com

Name Server: NS1.TARGETCO.COM
Name Server: NS2.TARGETCO.COM

DNSSEC: unsigned

Intelligence extracted:

Physical address (useful for physical penetration testing, social engineering)
Direct email addresses of three named individuals
Registrar (GoDaddy — relevant for domain hijacking attack surface)
DNSSEC status (unsigned → no DNSSEC protection → DNS hijacking more feasible)
Name servers hosting their own DNS (ns1.targetco.com → own DNS infrastructure)

GDPR Impact on WHOIS:

Since GDPR came into force (May 2018), most domain registrars have redacted personal information from public WHOIS for .com, .net, .org, and other gTLDs for registrants in the EU. This has significantly reduced the intelligence value of WHOIS for many domains. However:

Country-code TLDs (ccTLDs) like .uk, .au, .ca have varying GDPR compliance
US-based organizations often still have exposed WHOIS data
WHOIS history tools (SecurityTrails, DomainTools) may have pre-GDPR records
Whois for IP ranges (ARIN, RIPE, APNIC) is less affected by GDPR

WHOIS for IP Ranges:

Organizational IP ranges are registered with Regional Internet Registries (RIRs):

RIR	Region	Website
ARIN	North America	https://search.arin.net
RIPE NCC	Europe, Middle East, Central Asia	https://apps.db.ripe.net
APNIC	Asia-Pacific	https://wq.apnic.net
LACNIC	Latin America, Caribbean	https://lacnic.net
AFRINIC	Africa	https://afrinic.net

# IP WHOIS lookup
whois 203.0.113.50

# Response
NetRange: 203.0.113.0 - 203.0.113.255
CIDR: 203.0.113.0/24
NetName: TARGETCO-NET
NetHandle: NET-203-0-113-0-1
Parent: (NET-203-0-0-0-1)
NetType: Direct Assignment
Organization: Target Company Inc. (TCI-12)
OrgName: Target Company Inc.
OrgId: TCI-12
Address: 123 Corporate Drive
City: San Francisco
StateProv: CA
PostalCode: 94105
Country: US
OrgAbuseEmail: abuse@targetco.com
OrgTechEmail: noc@targetco.com

This reveals the organization's full IP range, enabling systematic scanning of all their registered IPs, and exposes NOC (Network Operations Center) and abuse contact emails — technical staff.

BGP Intelligence — Autonomous System Numbers

Large organizations own their own IP routing infrastructure, identified by an Autonomous System Number (ASN).

# Find ASN for organization
whois -h whois.bgp.he.net targetco.com

# Query BGP routing data
# https://bgp.he.net  ← Hurricane Electric BGP Toolkit
# https://bgpview.io  ← Visual BGP explorer

# Tool: asnmap
asnmap -a AS12345      # Get all IPs for an ASN
asnmap -org "Target Company"   # Find ASN by org name

Why ASN intelligence matters:

An organization's ASN reveals their entire IPv4 and IPv6 address space — every IP range they legitimately own globally. This is the most comprehensive method of identifying all the organization's Internet-facing IP space.

3.1.8 WHOIS Intelligence — Extracting Maximum Value

Beyond the basic contact information, WHOIS data contains several additional intelligence dimensions:

Domain Portfolio Discovery

Organizations typically own multiple domains — the primary domain, branded product domains, defensive registrations, and regional variations.

Reverse WHOIS: Finding all domains registered by the same registrant:

# Using DomainTools (commercial)
# Search by registrant email, name, or organization

# Using ViewDNS.info (free)
# https://viewdns.info/reversewhois/

# Using SecurityTrails
# https://securitytrails.com/list/registrant_email/it-admin@targetco.com

This can reveal:

Unreleased product domains
Acquisition targets (domains registered for companies being acquired)
Internal project names
Regional subsidiaries

Registrar Security Analysis

The domain registrar is a critical attack surface. Registrar account compromise enables:

DNS hijacking (changing NS records to attacker-controlled servers)
Domain transfer to attacker control
WHOIS email change (enabling password reset attacks on services)

Indicators of registrar security posture:

Registrar reputation (GoDaddy, Namecheap vs. enterprise registrars like MarkMonitor)
DNSSEC enabled (protects against certain DNS attacks)
Registrar lock status (prevents unauthorized transfers)
Privacy protection status

3.1.9 DNS Lookups — Lab-Level Practical Reference

This section provides a comprehensive, hands-on reference for DNS reconnaissance commands and workflows as used in professional lab environments.

Essential DNS Tools

dig (Domain Information Groper)

The primary command-line DNS tool. Highly flexible and returns detailed information.

# Basic A record lookup
dig targetco.com

# Specific record type
dig targetco.com MX
dig targetco.com NS
dig targetco.com TXT
dig targetco.com AAAA
dig targetco.com SOA
dig targetco.com CAA
dig targetco.com SRV

# Short output (answer only)
dig targetco.com +short

# All records (equivalent to ANY — not all servers honor this)
dig targetco.com ANY

# Trace the complete resolution path
dig targetco.com +trace

# Reverse lookup
dig -x 203.0.113.50

# Query a specific DNS server
dig @8.8.8.8 targetco.com        # Use Google's DNS
dig @1.1.1.1 targetco.com        # Use Cloudflare's DNS
dig @ns1.targetco.com targetco.com  # Query target's own nameserver

# Zone transfer attempt
dig axfr targetco.com @ns1.targetco.com

# DNS over HTTPS (bypasses local DNS filtering)
dig targetco.com @https://cloudflare-dns.com/dns-query

# DNSSEC verification
dig targetco.com +dnssec

nslookup

Simpler DNS tool, available on Windows and Linux.

# Basic lookup
nslookup targetco.com

# Specific record type
nslookup -type=MX targetco.com
nslookup -type=NS targetco.com
nslookup -type=TXT targetco.com
nslookup -type=ANY targetco.com

# Query specific server
nslookup targetco.com 8.8.8.8

# Interactive mode for zone transfer
nslookup
> server ns1.targetco.com
> set type=any
> targetco.com
> ls -d targetco.com   # Zone transfer in interactive mode

host

Simple, clean DNS lookup tool.

host targetco.com
host -t MX targetco.com
host -t NS targetco.com
host -a targetco.com    # All records
host 203.0.113.50       # Reverse lookup

DNS Reconnaissance Workflow — Complete Lab Procedure

#!/bin/bash
# Professional DNS Reconnaissance Script
DOMAIN="targetco.com"
OUTPUT_DIR="/tmp/dns_recon_${DOMAIN}"
mkdir -p $OUTPUT_DIR

echo "=== DNS RECONNAISSANCE: $DOMAIN ===" | tee $OUTPUT_DIR/summary.txt

# 1. Identify nameservers
echo "[*] Nameservers:" | tee -a $OUTPUT_DIR/summary.txt
dig $DOMAIN NS +short | tee $OUTPUT_DIR/nameservers.txt

# 2. SOA record (admin email)
echo "[*] SOA Record:" | tee -a $OUTPUT_DIR/summary.txt
dig $DOMAIN SOA +short | tee $OUTPUT_DIR/soa.txt

# 3. A records
echo "[*] A Records:" | tee -a $OUTPUT_DIR/summary.txt
dig $DOMAIN A +short | tee $OUTPUT_DIR/a_records.txt

# 4. MX records (email infrastructure)
echo "[*] MX Records:" | tee -a $OUTPUT_DIR/summary.txt
dig $DOMAIN MX +short | tee $OUTPUT_DIR/mx_records.txt

# 5. TXT records (SPF, DMARC, verification tokens)
echo "[*] TXT Records:" | tee -a $OUTPUT_DIR/summary.txt
dig $DOMAIN TXT +short | tee $OUTPUT_DIR/txt_records.txt

# 6. DMARC policy
echo "[*] DMARC:" | tee -a $OUTPUT_DIR/summary.txt
dig _dmarc.$DOMAIN TXT +short | tee $OUTPUT_DIR/dmarc.txt

# 7. DKIM (try common selectors)
for selector in default google mail k1 selector1 selector2; do
    result=$(dig ${selector}._domainkey.$DOMAIN TXT +short)
    if [ ! -z "$result" ]; then
        echo "[*] DKIM Selector: $selector" | tee -a $OUTPUT_DIR/summary.txt
        echo "$result" | tee -a $OUTPUT_DIR/dkim.txt
    fi
done

# 8. Zone transfer attempts
echo "[*] Attempting zone transfers..." | tee -a $OUTPUT_DIR/summary.txt
while read ns; do
    echo "[*] Trying AXFR from $ns..." | tee -a $OUTPUT_DIR/summary.txt
    dig axfr $DOMAIN @$ns | tee $OUTPUT_DIR/axfr_${ns}.txt
done < $OUTPUT_DIR/nameservers.txt

# 9. IPv6
echo "[*] AAAA Records:" | tee -a $OUTPUT_DIR/summary.txt
dig $DOMAIN AAAA +short | tee $OUTPUT_DIR/aaaa_records.txt

# 10. CAA
echo "[*] CAA Records:" | tee -a $OUTPUT_DIR/summary.txt
dig $DOMAIN CAA +short | tee $OUTPUT_DIR/caa.txt

echo "[*] DNS Reconnaissance Complete. Results in $OUTPUT_DIR"

Online DNS Reconnaissance Tools

Tool	URL	Best For
DNSDumpster	https://dnsdumpster.com	Visual DNS map, subdomains, MX, TXT
SecurityTrails	https://securitytrails.com	DNS history, all record types
MXToolbox	https://mxtoolbox.com	Email infrastructure, MX, SPF, DMARC analysis
DNSlytics	https://dnslytics.com	Reverse DNS, related domains
ViewDNS.info	https://viewdns.info	Comprehensive DNS tools
IntoDNS	https://intodns.com	DNS health check and misconfiguration detection
DMARC Inspector	https://dmarcian.com/dmarc-inspector/	DMARC policy analysis
MXToolbox DKIM Checker	https://mxtoolbox.com/dkim.aspx	DKIM record analysis
ARIN	https://search.arin.net	IP WHOIS, ASN lookups
BGP.he.net	https://bgp.he.net	BGP routing, ASN details

3.1.10 Cloud vs. Self-Hosted Applications and Related Subdomains

Modern organizations rarely run entirely self-hosted infrastructure. Understanding the distinction between cloud-hosted and self-hosted assets dramatically affects the penetration testing approach, authorization requirements, and vulnerability surface.

Identifying Cloud-Hosted Infrastructure

Method 1: DNS CNAME Analysis

CNAME records pointing to cloud provider domains are the clearest indicator of cloud hosting:

CNAME Destination	Cloud Service
`*.cloudfront.net`	AWS CloudFront CDN
`*.awsglobalaccelerator.com`	AWS Global Accelerator
`*.elasticbeanstalk.com`	AWS Elastic Beanstalk
`*.s3.amazonaws.com`	AWS S3
`.s3-website-.amazonaws.com`	AWS S3 Static Website
`*.azurewebsites.net`	Azure App Service
`*.azurefd.net`	Azure Front Door
`*.blob.core.windows.net`	Azure Blob Storage
`*.trafficmanager.net`	Azure Traffic Manager
`*.appspot.com`	Google App Engine
`*.run.app`	Google Cloud Run
`*.cloudfunctions.net`	Google Cloud Functions
`*.storage.googleapis.com`	Google Cloud Storage
`.herokussl.com` / `.herokudns.com`	Heroku
`*.netlify.app`	Netlify
`*.vercel.app`	Vercel
`*.github.io`	GitHub Pages
`*.pages.dev`	Cloudflare Pages
`*.fastly.net`	Fastly CDN
`*.edgekey.net`	Akamai
`*.cdn.cloudflare.net`	Cloudflare CDN

Method 2: IP Range Identification

Cloud providers publish their IP ranges. Identifying that a target IP belongs to a cloud provider IP range reveals cloud hosting:

AWS IP ranges: https://ip-ranges.amazonaws.com/ip-ranges.json
Azure IP ranges: https://download.microsoft.com/download/7/1/D/71D86715-5596-4529-9B13-DA13A5DE5B63/ServiceTags_Public.json
GCP IP ranges: https://cloud.google.com/compute/docs/faq#find_ip_range
Cloudflare IP ranges: https://cloudflare.com/ips/

# Check if IP belongs to AWS
curl https://ip-ranges.amazonaws.com/ip-ranges.json | python3 -c "
import json, sys
data = json.load(sys.stdin)
target_ip = '203.0.113.50'
for prefix in data['prefixes']:
    import ipaddress
    if ipaddress.ip_address(target_ip) in ipaddress.ip_network(prefix['ip_prefix']):
        print(f'AWS Region: {prefix[\"region\"]}, Service: {prefix[\"service\"]}')
"

Method 3: HTTP Response Headers

HTTP headers often reveal cloud provider and CDN usage:

curl -I https://www.targetco.com

# Headers revealing cloud services:
# X-Amz-Cf-Id: → CloudFront
# CF-Ray: → Cloudflare
# X-Azure-Ref: → Azure Front Door
# X-GUploader-UploadID: → Google Cloud Storage
# X-Served-By: cache-... → Fastly
# Via: 1.1 akamai → Akamai
# Server: AmazonS3 → AWS S3
# X-Powered-By: Express on Google Cloud → GCP

Subdomain Enumeration — Professional Techniques

Subdomain discovery is one of the highest-value activities in passive reconnaissance. Modern enterprise organizations have hundreds or thousands of subdomains, many of which are forgotten, unmanaged, or running legacy software.

Certificate Transparency (CT) Logs:

Every SSL/TLS certificate issued by a trusted CA is logged to public Certificate Transparency logs. These logs contain the domain names (including subdomains) in every certificate issued.

# crt.sh — The primary CT log search interface
curl 'https://crt.sh/?q=%.targetco.com&output=json' | \
python3 -c "import json,sys; [print(e['name_value']) for e in json.load(sys.stdin)]" | \
sort -u

# Subfinder — Comprehensive passive subdomain discovery
subfinder -d targetco.com -all -recursive

# Amass — Comprehensive subdomain enumeration
amass enum -passive -d targetco.com
amass enum -active -d targetco.com

# assetfinder — Fast, focused subdomain discovery
assetfinder targetco.com

# chaos — Project Discovery's subdomain database
chaos -d targetco.com

Why CT logs are the most valuable passive subdomain source:

Cover all publicly trusted SSL certificates since approximately 2013
Include certificates for subdomains that may no longer resolve (revealing historical infrastructure)
Include wildcard certificates (*.targetco.com) that hint at dynamic subdomain usage
Include certificates for internal systems that accidentally got public certs
Are 100% passive — no interaction with the target

Subdomain Takeover:

Subdomain takeover is a vulnerability where a subdomain's DNS record points to a third-party service that is no longer configured for that subdomain, allowing an attacker to claim the subdomain.

DNS: dev.targetco.com → CNAME → targetco.herokuapp.com (Heroku)
If targetco has cancelled their Heroku account, targetco.herokuapp.com is unclaimed.
Attacker creates a Heroku app at targetco.herokuapp.com
Attacker now controls dev.targetco.com

This enables:

Serving malicious content under the target's legitimate domain
Stealing cookies scoped to *.targetco.com
Bypassing Content Security Policy (CSP)
Credential phishing under a trusted domain

# Check for subdomain takeover vulnerabilities
subjack -w discovered_subdomains.txt -t 100 -timeout 30 -o results.txt -ssl
nuclei -l discovered_subdomains.txt -t takeovers/

Cloud-Specific Subdomain Intelligence:

AWS S3 Buckets:
S3 bucket names in URLs often follow predictable patterns:

https://targetco-assets.s3.amazonaws.com
https://s3.amazonaws.com/targetco-backups
https://targetco.s3-website-us-east-1.amazonaws.com

Tools for S3 bucket discovery:

# S3Scanner
s3scanner scan --bucket targetco
s3scanner scan --bucket-file probable_bucket_names.txt

# AWS CLI (if credentials available)
aws s3 ls s3://targetco-assets --no-sign-request

# lazys3
ruby lazys3.rb targetco

Common bucket misconfigurations:

Public read access (anyone can list and download files)
Public write access (anyone can upload — potential for malicious content)
Exposed bucket policy showing other IAM principals with access

3.1.11 Social Media Scraping

Social media is one of the most underestimated intelligence sources in professional penetration testing. Employees voluntarily and publicly disclose enormous amounts of information relevant to security assessments.

LinkedIn — The Primary Corporate Intelligence Source

LinkedIn is the most valuable social media platform for penetration testing reconnaissance because it is specifically designed for professional networking and intentionally exposes professional information.

Intelligence Categories from LinkedIn:

1. Employee Enumeration

Search: "Target Company Inc" employees
Result: 847 employees found

Filter by:
- Department: Information Technology (reveals IT staff count and roles)
- Location (reveals office locations)
- Seniority: Entry Level / Associate (reveals junior staff who may be social engineering targets)

High-value employee targets:

IT administrators (system access)
Developers (code access, internal tools)
Security team members (defenses, tools in use)
C-suite executives (high-credibility email targets)
Finance staff (wire transfer fraud targets)
Receptionists and administrative staff (physical access social engineering)
Help desk staff (password reset social engineering)

2. Technology Stack Discovery from Job Listings

Job postings are extraordinarily revealing because they list exactly what technologies are in use:

Job Title: Senior DevOps Engineer at Target Company

Requirements:
• 5+ years experience with AWS (EC2, EKS, RDS, S3, Lambda, VPC, CloudWatch)
• Strong proficiency with Terraform for infrastructure as code
• Experience with Kubernetes and Helm chart deployment
• Familiarity with CI/CD pipelines (Jenkins, GitLab CI, or CircleCI)
• HashiCorp Vault for secrets management
• Elasticsearch, Kibana, and Logstash (ELK stack) for log management
• Experience with Prometheus and Grafana for monitoring
• PostgreSQL and Redis database management
• Proficiency with Docker containerization

This single job listing reveals:

Cloud: AWS (with specific services identified)
IaC: Terraform
Container orchestration: Kubernetes + Helm
CI/CD: Jenkins, GitLab CI, or CircleCI
Secrets management: HashiCorp Vault
Logging: ELK stack
Monitoring: Prometheus + Grafana
Databases: PostgreSQL + Redis
Containerization: Docker

Every one of these technologies has known vulnerabilities and misconfigurations that can be specifically targeted.

3. Individual Employee Profile Intelligence

John Smith — Senior Network Engineer, Target Company Inc.

Current: "Leading network security initiative migrating from on-prem Cisco ASA 
         firewalls to Palo Alto NGFW with Panorama management. Also managing 
         our SD-WAN deployment with VMware VeloCloud."

Skills: Cisco ASA, Palo Alto Networks, Panorama, SD-WAN, VeloCloud, 
        BGP, OSPF, MPLS, Wireshark, SolarWinds

"Excited to be presenting at Cisco Live 2024 on our journey to 
SD-WAN! Our office locations in San Francisco, Austin, and London 
are all now connected."

This single profile reveals:

Firewall technology (Cisco ASA → Palo Alto NGFW transition in progress)
Specific management platform (Panorama)
SD-WAN vendor (VMware VeloCloud)
All office locations (San Francisco, Austin, London)
Network protocols in use (BGP, OSPF, MPLS)
Monitoring tools (SolarWinds)
The engineer's real name and potentially their email address

LinkedIn Search Techniques:

# Site-specific Google searches for LinkedIn
site:linkedin.com/in "Target Company" "network engineer"
site:linkedin.com/in "Target Company" "security"
site:linkedin.com/in "Target Company" "developer"
site:linkedin.com/in "Target Company" "IT manager"

# LinkedIn Sales Navigator (premium) — most powerful
# Full employee lists, contact info, organizational charts

LinkedIn Reconnaissance Tools:

# LinkedInt — LinkedIn intelligence gathering
python LinkedInt.py -u your_linkedin_account -p password -k "Target Company"

# CrossLinked — Employee enumeration via LinkedIn
python crosslinked.py -f '{first}.{last}@targetco.com' "Target Company"
# This both finds employee names AND constructs likely email addresses

# linkedin2username — Generate username lists from LinkedIn scraping
python linkedin2username.py -c "Target Company" -u your_account -p password

Twitter / X Intelligence

Twitter (X) provides real-time organizational intelligence:

What employees tweet about:

Technology outages ("our Splunk is down again...")
Security incidents ("just blocked a phishing campaign targeting @targetco")
New technology deployments ("just pushed our first workload to AWS!")
Company news and events
Personal information (vacation dates → reduced staffing)
Conference attendance (away from office)
Complaints about internal tools (revealing technology names)

# Twitter OSINT tools
twint -u johndoe_sysadmin --tweets  # Scrape all tweets without API
twint -s "targetco.com" --lang en   # Search tweets mentioning the target
twint -u johndoe_sysadmin -o output.json --json

# Advanced Twitter search operators
site:twitter.com "Target Company" security breach
site:twitter.com "targetco.com" password

GitHub and Code Repository Intelligence

GitHub is one of the highest-value passive recon sources for technical intelligence. Developers commit sensitive information to public repositories constantly.

What to search for on GitHub:

# GitHub Search — most powerful passive recon tool for technical data

# Organization-specific searches
site:github.com "targetco.com"
site:github.com "targetco" "password"
site:github.com "targetco" "api_key"
site:github.com "targetco" "secret"
site:github.com "targetco" "internal"

# GitHub native search
org:targetco                           # All repos in org
org:targetco filename:.env             # .env files (credentials)
org:targetco "BEGIN RSA PRIVATE KEY"  # Private keys
org:targetco "AKIA"                    # AWS Access Key IDs (start with AKIA)
org:targetco "mongodb://",            # Database connection strings
org:targetco "postgresql://",
org:targetco "mysql://"

# GitHub Dorks (search patterns for sensitive data)
"targetco.com" password
"targetco.com" secret
"targetco.com" token
"targetco.com" api_key
"targetco.com" private_key
"@targetco.com" password

GitHub OSINT Tools:

# truffleHog — Searches git history for secrets
trufflehog github --org targetco --json

# GitLeaks — Audit git repos for secrets
gitleaks detect --source /path/to/cloned/repo -v

# Gitrob — Reconnaissance on GitHub organizations
gitrob analyze targetco

# git-secrets — Prevent credential commits (defensive, but reveals what to look for)

What developers accidentally commit:

Sensitive Data Type	Example
AWS credentials	`AKIAIOSFODNN7EXAMPLE / wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY`
API keys	`stripe_key = "------------------------"`
Database credentials	`DB_PASS=Production_P@ssw0rd_2024`
Private SSH keys	`-----BEGIN RSA PRIVATE KEY-----`
SSL private keys	`-----BEGIN PRIVATE KEY-----`
OAuth tokens	`-------------------------`
JWT secrets	`--------------------------`
Internal IP addresses	Connection strings with internal IPs
Cloud configuration files	Terraform state files with resource details

3.1.12 Employee Intelligence Gathering

Employee intelligence gathering synthesizes social media, professional profiles, breach data, and public records to build detailed profiles of target organization personnel.

Building the Target Employee Profile

For each high-value employee target, a professional assessment builds:

The Target Profile Template:

Employee Profile: John Smith
===========================
Current Role: Senior Systems Administrator, Target Company Inc.
LinkedIn: linkedin.com/in/john-smith-sysadmin
Twitter: @jsmith_sysadmin

Contact Information:
- Work email: j.smith@targetco.com (derived from email format)
- Personal email: johnsmith1982@gmail.com (from breach data)
- Phone: +1-415-555-1234 (from conference registration, LinkedIn)

Technical Skills (from LinkedIn, GitHub, job listings):
- Windows Server 2016/2019/2022
- Active Directory, Group Policy, SCCM
- VMware vSphere 7.0
- PowerShell scripting
- Backup: Veeam

Personal Information (for social engineering pretext):
- Alma mater: University of California, Berkeley (from LinkedIn)
- Previous employer: CloudBase Inc. (from LinkedIn)
- Hobbies: cycling, homebrewing (from Twitter)
- Location: San Francisco, CA

Exposure Assessment:
- Appears in 3 data breaches (HaveIBeenPwned): LinkedIn breach, Adobe breach, Tumblr breach
- Leaked password hash (LinkedIn 2012 breach): "LinkedInPasswordHash"
- Password reuse risk: HIGH (common for accounts 10+ years old)

Social Engineering Attack Vectors:
1. IT Help Desk impersonation: "Hi John, this is Sarah from the IT help desk..."
2. Vendor impersonation: "Hi, this is VMware support calling about your license renewal..."
3. Password reset phishing: Custom email targeting @targetco.com with realistic IT branding
4. Vishing (voice phishing): Calling directly using work context established from LinkedIn

Email Address Format Discovery

Before email addresses can be used in social engineering or checked against breach databases, the organization's email format must be determined.

Method 1: Hunter.io

# Hunter.io API
curl "https://api.hunter.io/v2/domain-search?domain=targetco.com&api_key=YOUR_KEY"

# Response
{
  "data": {
    "domain": "targetco.com",
    "organization": "Target Company Inc.",
    "pattern": "{first}.{last}",    ← EMAIL FORMAT DISCOVERED
    "emails": [
      {"value": "john.smith@targetco.com", "type": "personal"},
      {"value": "jane.doe@targetco.com", "type": "personal"},
      {"value": "security@targetco.com", "type": "generic"}
    ]
  }
}

Common email formats:

Format	Example
`{first}.{last}@domain.com`	john.smith@targetco.com
`{first}{last}@domain.com`	johnsmith@targetco.com
`{f}{last}@domain.com`	jsmith@targetco.com
`{first}_{last}@domain.com`	john_smith@targetco.com
`{first}@domain.com`	john@targetco.com
`{last}{first}@domain.com`	smithjohn@targetco.com
`{f}.{last}@domain.com`	j.smith@targetco.com

Method 2: Email Format from Found Addresses

If even one employee's email is confirmed (from a breach database, email header, or other source), the format is revealed and can be applied to all other employee names.

Method 3: CrossLinked

# CrossLinked: scrapes LinkedIn employees and generates email addresses
python3 crosslinked.py -f '{first}.{last}@targetco.com' "Target Company"
# Outputs: john.smith@targetco.com, jane.doe@targetco.com, etc.

Method 4: Email Verification

Discovered email addresses can be verified (without sending an email) by:

# smtp-user-enum — Verifies email existence via SMTP
smtp-user-enum -M VRFY -U users.txt -t mail.targetco.com
smtp-user-enum -M EXPN -U users.txt -t mail.targetco.com
smtp-user-enum -M RCPT -U users.txt -t mail.targetco.com

# Note: VRFY and EXPN are often disabled on modern mail servers for security

3.1.13 Cryptographic Flaws

Cryptographic weaknesses discovered during passive reconnaissance provide direct attack vectors during the active exploitation phase. Understanding cryptographic flaws at a senior level requires deep knowledge of both the theoretical weaknesses and the practical exploitation techniques.

Why Cryptographic Analysis Belongs in Passive Reconnaissance

Passive reconnaissance can reveal cryptographic weaknesses without touching the target:

SSL/TLS certificate details are publicly visible
Certificate Transparency logs reveal certificate history
SSL test services (SSL Labs) provide detailed analysis of TLS configurations
DNS DMARC and DKIM records reveal email cryptographic configuration

SSL/TLS Protocol Weaknesses

Protocol Version Vulnerabilities:

Protocol	Version	Status	Known Attacks
SSL 2.0	Ancient	PROHIBITED	DROWN, complete deprecation
SSL 3.0	Ancient	PROHIBITED	POODLE
TLS 1.0	RFC 2246 (1999)	DEPRECATED	BEAST, POODLE (in CBC mode)
TLS 1.1	RFC 4346 (2006)	DEPRECATED	BEAST (partial)
TLS 1.2	RFC 5246 (2008)	CURRENT	Various cipher suite weaknesses
TLS 1.3	RFC 8446 (2018)	RECOMMENDED	No known practical attacks

TLS 1.0 and 1.1 Deprecation:

The IETF formally deprecated TLS 1.0 and TLS 1.1 in RFC 8996 (March 2021)
PCI DSS v3.2+ requires disabling TLS 1.0 for all in-scope systems
NIST SP 800-52 Rev 2 prohibits TLS 1.0 and 1.1 for federal systems
Finding TLS 1.0 or 1.1 support is a compliance finding in addition to a technical vulnerability

Cipher Suite Weaknesses:

A cipher suite specifies the algorithms used for key exchange, authentication, encryption, and integrity checking. Weak cipher suites are a common finding.

Cipher Suite Format:

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
 │    │     │      │        │       │
 │    │     │      │        │       └── MAC/PRF algorithm (SHA384)
 │    │     │      │        └── Cipher mode (GCM = authenticated)
 │    │     │      └── Encryption algorithm and key size (AES-256)
 │    │     └── Authentication algorithm (RSA)
 │    └── Key exchange algorithm (ECDHE = Elliptic Curve Diffie-Hellman Ephemeral)
 └── Protocol

Weak/Deprecated Cipher Suites:

Cipher Suite Issue	Specific Problem	Attack
NULL cipher	No encryption	Plaintext exposure
EXPORT ciphers	Deliberately weakened (40-56 bit)	FREAK, LOGJAM
RC4	Stream cipher with statistical biases	RC4 NOMORE
DES/3DES	56-bit DES key, 112-bit 3DES	SWEET32
RSA key exchange (no forward secrecy)	Static key exchange	Historical traffic decryption if key compromised
MD5 for signatures	Collision vulnerabilities	Certificate forgery
SHA1 for signatures	Collision vulnerabilities	SHAttered
Anonymous DH (aDH/aNULL)	No server authentication	MitM

Forward Secrecy (Perfect Forward Secrecy — PFS):

Forward secrecy means that session keys are not compromised even if the server's long-term private key is compromised. It is provided by Ephemeral key exchange algorithms:

ECDHE (Elliptic Curve Diffie-Hellman Ephemeral) — Preferred
DHE (Diffie-Hellman Ephemeral) — Acceptable, but slower than ECDHE
RSA key exchange — No forward secrecy (static keys)

Identification of TLS weaknesses during passive recon:

# SSL Labs — Most comprehensive TLS analysis (passive — uses their servers)
# https://www.ssllabs.com/ssltest/analyze.html?d=targetco.com

# testssl.sh — Active tool but usable against own infrastructure
testssl.sh --full targetco.com

# sslscan — Active scanning tool
sslscan targetco.com

# nmap TLS scripts (active)
nmap --script ssl-enum-ciphers -p 443 targetco.com
nmap --script ssl-dh-params -p 443 targetco.com

# Check for HSTS
curl -I https://targetco.com | grep -i strict

Major TLS Vulnerabilities — Professional Reference

POODLE — Padding Oracle On Downgraded Legacy Encryption

CVE: CVE-2014-3566
Affects: SSL 3.0 (original POODLE), TLS 1.0/1.1 (POODLE TLS variant)
Mechanism: Exploits CBC padding oracle in SSL 3.0 to decrypt sessions
Impact: Decryption of encrypted HTTP cookies, potentially exposing session tokens
Mitigation: Disable SSL 3.0 and TLS 1.0; use TLS 1.2+ with authenticated encryption modes

BEAST — Browser Exploit Against SSL/TLS

CVE: CVE-2011-3389
Affects: TLS 1.0 using CBC mode cipher suites
Mechanism: Exploits a predictable IV (Initialization Vector) in TLS 1.0 CBC mode
Impact: Potential decryption of HTTPS traffic
Mitigation: TLS 1.2+ with AEAD ciphers (GCM mode); or RC4 (itself now deprecated)

HEARTBLEED

CVE: CVE-2014-0160
Affects: OpenSSL 1.0.1 through 1.0.1f
Mechanism: Buffer over-read in the HeartBeat extension — allows reading 64KB of server memory per request
Impact: Exposure of private keys, session tokens, passwords from server memory
Mitigation: Upgrade to OpenSSL 1.0.1g or later; revoke and reissue all certificates

# Check for Heartbleed (active)
nmap --script ssl-heartbleed -p 443 targetco.com

FREAK — Factoring RSA Export Keys

CVE: CVE-2015-0204
Affects: Servers supporting RSA EXPORT cipher suites (FREAK)
Mechanism: Client can be forced to use deliberately weakened 512-bit export RSA keys, which can be factored
Impact: Full session decryption via man-in-the-middle
Mitigation: Disable all EXPORT cipher suites

LOGJAM

CVE: CVE-2015-4000
Affects: Servers supporting DHE EXPORT cipher suites (512-bit DH)
Mechanism: Similar to FREAK but targeting Diffie-Hellman
Impact: Downgrade to weak DH parameters, enabling session decryption
Mitigation: Disable DHE EXPORT; use 2048-bit+ DH parameters or ECDHE

DROWN — Decrypting RSA with Obsolete and Weakened eNcryption

CVE: CVE-2016-0800
Affects: Any server sharing an RSA private key with a server that supports SSL 2.0
Mechanism: Uses SSL 2.0 vulnerabilities to decrypt TLS sessions
Impact: Decryption of TLS sessions for affected servers
Mitigation: Disable SSL 2.0 on all servers sharing a private key

SWEET32 — Birthday Attacks on 64-bit Block Ciphers

CVE: CVE-2016-2183
Affects: 3DES (64-bit block cipher) in TLS
Mechanism: Birthday attack — after ~32GB of same-key traffic, collision reveals plaintext
Impact: Session cookie exposure in long-lived HTTPS sessions
Mitigation: Disable 3DES (RC4_128, 3DES_EDE_CBC) cipher suites; limit session renegotiation

Certificate Weaknesses

Weak Key Sizes:

Algorithm	Minimum Recommended	Deprecated Below
RSA	2048 bits	1024 bits (since 2013)
ECDSA	256 bits (P-256)	Below P-256
DSA	2048 bits	1024 bits

Signature Algorithm Weaknesses:

MD5 signatures: Cryptographically broken since 2004; MD5 signed certificates should be immediately revoked
SHA-1 signatures: Theoretically broken (SHAttered attack, 2017); browsers since 2017 reject SHA-1 certificates
SHA-2 (SHA-256, SHA-384, SHA-512): Current standard, no known practical weaknesses
SHA-3: Available but rarely deployed; provides additional algorithm diversity

Certificate Validity and Management Issues:

Expired certificates: Indicates poor certificate management, potentially revealing processes for certificate deployment that can be abused
Self-signed certificates: Indicates non-standard deployment; may indicate test or internal systems exposed publicly
Wildcard certificates (*.targetco.com): A single wildcard certificate covers all subdomains; if the private key is compromised, ALL subdomains are compromised
Overly broad SAN lists: Many subdomains in SAN list reveals infrastructure (valuable for recon) and if one subjectAltName system is compromised, the certificate trust may be leveraged

Certificate Authority Trust Issues:

Unknown/private CA: Certificate signed by an internal CA — server may only be intended for internal use but is externally exposed
Distrusted CA: A CA whose root certificate has been revoked or distrusted by major browsers (e.g., Symantec CAs distrusted in 2018)

3.1.14 Finding Information from SSL Certificates

SSL/TLS certificates contain rich metadata that is extremely valuable for passive reconnaissance. Every certificate presented by a server is a public document and can be examined without any authentication.

Reading Certificate Information

Using command-line tools:

# Get certificate from server
openssl s_client -connect targetco.com:443 -servername targetco.com < /dev/null 2>/dev/null | \
openssl x509 -noout -text

# Extract just the key fields
openssl s_client -connect targetco.com:443 < /dev/null 2>/dev/null | \
openssl x509 -noout -subject -issuer -dates -fingerprint -ext subjectAltName

# Output example:
subject=CN=targetco.com, O=Target Company Inc., L=San Francisco, ST=California, C=US
issuer=CN=DigiCert TLS RSA SHA256 2020 CA1, O=DigiCert Inc, C=US
notBefore=Jan  1 00:00:00 2024 GMT
notAfter=Dec 31 23:59:59 2024 GMT
SHA256 Fingerprint=AA:BB:CC:DD:EE:FF:...
X509v3 Subject Alternative Name:
    DNS:targetco.com
    DNS:www.targetco.com
    DNS:api.targetco.com
    DNS:auth.targetco.com
    DNS:mail.targetco.com
    DNS:dev.targetco.com
    DNS:staging.targetco.com
    DNS:internal-wiki.targetco.com   ← INTERNAL SYSTEM EXPOSED!
    DNS:jira.targetco.com
    DNS:confluence.targetco.com
    DNS:gitlab.targetco.com

Intelligence from this single certificate:

Organization: "Target Company Inc." — confirms organization name
Location: San Francisco, California, USA — physical location confirmation
CA: DigiCert — enterprise-grade certificate provider
Validity period: Full year certificate (common in enterprise)
All subdomains (SAN list):
- api.targetco.com — API endpoint exists
- auth.targetco.com — Authentication service (SSO? OAuth? SAML?)
- dev.targetco.com — Development environment externally exposed
- staging.targetco.com — Staging environment externally exposed
- internal-wiki.targetco.com — CRITICAL: Internal wiki publicly accessible
- jira.targetco.com — Jira project management
- confluence.targetco.com — Confluence wiki
- gitlab.targetco.com — Internal GitLab instance

This single certificate reveals an entire secondary set of attack targets.

Certificate Transparency (CT) Log Mining

Certificate Transparency is a public audit trail for SSL certificates. Every certificate issued by a publicly trusted CA is logged to append-only CT logs, and these logs are permanently accessible.

Why CT logs provide passive historical intelligence:

Every certificate ever issued for a domain is logged — even expired ones
Reveals subdomains that existed in the past (even if the DNS records are now gone)
Shows when the organization changed certificate providers
Reveals internal project names in historically issued certificates
Shows wildcard vs. specific subdomain certificate usage patterns

CT Log Tools:

# crt.sh — Primary public CT search interface
curl 'https://crt.sh/?q=%.targetco.com&output=json' 2>/dev/null | \
python3 -c "
import json, sys
data = json.load(sys.stdin)
domains = set()
for cert in data:
    names = cert.get('name_value', '').split('\n')
    for name in names:
        name = name.strip().lstrip('*.')
        if name:
            domains.add(name)
for d in sorted(domains):
    print(d)
" | sort -u > ct_subdomains.txt

# Certspotter
curl "https://api.certspotter.com/v1/issuances?domain=targetco.com&include_subdomains=true&expand=dns_names" \
-H "Authorization: Bearer YOUR_TOKEN" | python3 -m json.tool

# Facebook CT Monitoring
curl "https://graph.facebook.com/certificates?query=targetco.com&fields=domains,issuer_name,not_before,not_after"

Analyzing CT History for Intelligence:

Historical certificates for targetco.com:

2015: *.targetco.com (wildcard) — old infrastructure approach
2018: targetco.com, www.targetco.com, api.targetco.com — legacy structure
2020: includes dev.targetco.com — development environment appears
2021: includes vpn.targetco.com — VPN service added
2022: includes staging.targetco.com, qa.targetco.com — test environments
2023: internal-wiki.targetco.com DISAPPEARS from certificate — removed? still running?
2024: current certificate as analyzed above

Insight: dev, staging, and qa environments have existed since 2020-2022 and likely 
still run legacy software from when they were first deployed.

Certificate Analysis Tools

Tool	Purpose	URL/Command
SSL Labs	Comprehensive TLS analysis with grade	https://www.ssllabs.com/ssltest/
crt.sh	CT log search	https://crt.sh
Censys	Certificate search, host analysis	https://search.censys.io
Observatory by Mozilla	TLS, headers, security analysis	https://observatory.mozilla.org
testssl.sh	Command-line TLS testing	`testssl.sh targetco.com`
cert-parse	Extract SANs from certificates	`openssl s_client ...
tlsx	Fast TLS scanning	{% raw %}`tlsx -u targetco.com -san -cn`

# tlsx — Modern, fast TLS reconnaissance
tlsx -u targetco.com -san -cn -resp -p 443,8443
tlsx -list domains.txt -san -json -o tls_results.json

3.1.15 Company Reputation and Security Posture

Assessing a target organization's public security reputation and posture provides context for the assessment and reveals historical vulnerabilities, threat actor attention, and security program maturity.

Threat Intelligence Platforms

VirusTotal

VirusTotal aggregates results from 70+ antivirus engines and URL/file scanners. It is valuable for:

# Check domain reputation
curl "https://www.virustotal.com/api/v3/domains/targetco.com" \
-H "x-apikey: YOUR_KEY"

# Check IP reputation
curl "https://www.virustotal.com/api/v3/ip_addresses/203.0.113.50" \
-H "x-apikey: YOUR_KEY"

Intelligence gathered:

Malicious URL reports (has the domain served malware?)
Phishing reports (has the domain been used in phishing?)
Malware downloads (has the domain distributed malware?)
Associated files (malware samples communicating with the IP)
Community comments (security researcher observations)

A domain with a history of malware distribution may indicate previous compromise or insider threat activity — both critical pre-assessment intelligence.

Shodan (Reputation/History):

Beyond its scanning capabilities (covered in 3.1.21), Shodan maintains historical data on every IP's service history:

What ports were open in the past
What software versions were running
When services appeared and disappeared

Censys:

Similar to Shodan, Censys regularly scans the entire Internet's IPv4 address space.

# Censys search for organization
curl "https://search.censys.io/api/v2/hosts/search" \
-H "Authorization: Basic $(echo -n 'ID:SECRET' | base64)" \
-d '{"q": "autonomous_system.organization: \"Target Company Inc.\"", "per_page": 100}'

AlienVault OTX (Open Threat Exchange):

# Check IP in threat intelligence database
curl "https://otx.alienvault.com/api/v1/indicators/IPv4/203.0.113.50/general" \
-H "X-OTX-API-KEY: YOUR_KEY"

# Check domain
curl "https://otx.alienvault.com/api/v1/indicators/domain/targetco.com/general" \
-H "X-OTX-API-KEY: YOUR_KEY"

Intelligence gathered:

Pulse reports (threat actor activity associated with the IP/domain)
Reputation score
Associated malware families
Geographic threat context

Paste Sites and Dark Web Monitoring

Pastebin and Public Paste Sites:

Attackers and insiders frequently post stolen data or tools to public paste sites:

# Search paste sites for target-related content
# Manual search:
site:pastebin.com "targetco.com"
site:paste.ee "targetco.com"
site:ghostbin.com "targetco.com"
site:hastebin.com "targetco.com"
site:controlc.com "targetco.com"

# Automated tool
pastehunter --query "targetco.com" --output /tmp/paste_results.json

Public Vulnerability Databases

CVE Correlation:

Once technology stack is identified (from job listings, SSL cert analysis, etc.), cross-reference against CVE databases:

# Search NVD for vendor vulnerabilities
curl "https://services.nvd.nist.gov/rest/json/cves/2.0?keywordSearch=Apache+Tomcat+9.0.41"

# Shodan CVE search
shodan search "org:\"Target Company\" vuln:cve-2021-44228"   # Log4Shell

# Search Exploit-DB for public exploits
searchsploit apache tomcat 9.0.41
searchsploit --json apache tomcat | python3 -m json.tool

Bug Bounty Program Analysis

Checking whether the target has a public bug bounty program reveals:

What scope they consider important (scope defines what they want tested)
What has already been found by bug bounty hunters
The organization's security program maturity
Types of vulnerabilities that have been paid out (suggesting they exist)

# Bug bounty programs
# HackerOne: https://hackerone.com/directory/programs
# Bugcrowd: https://bugcrowd.com/programs
# Intigriti: https://www.intigriti.com/programs

# Search for target's program
site:hackerone.com "Target Company"
site:bugcrowd.com "Target Company"

Public Disclosure Analysis:

Bug bounty platforms publish disclosed vulnerability reports after patching. Search for disclosures related to the target:

site:hackerone.com/reports "targetco.com"
site:hackerone.com/reports "Target Company"

Disclosed reports reveal:

Types of vulnerabilities that have been found historically
Specific applications and APIs that have had vulnerabilities
The organization's patch response speed
Vulnerability classes the security team may have blind spots for

3.1.16 File Metadata

Files published on an organization's website contain embedded metadata that can expose sensitive information about the organization's internal infrastructure, personnel, and software versions.

What is File Metadata?

File metadata is data embedded within a file that describes the file itself. Users creating and publishing files typically are unaware of the metadata their files contain.

Types of metadata by file format:

File Type	Metadata Format	What It Contains
PDF	XMP, Dublin Core, Application-specific	Author, software, company, internal paths, modification history
Word (.docx)	Office Open XML metadata	Author, company, last modified by, document history, template path
Excel (.xlsx)	Office Open XML metadata	Author, company, username, formula history
PowerPoint (.pptx)	Office Open XML metadata	Presentation author, company, internal links
JPEG/PNG	EXIF	GPS coordinates, camera model, date/time, software
TIFF	EXIF + IPTC	Same as JPEG plus additional photo metadata
MP4/Video	Various	Creation software, GPS, encoding software

What Metadata Reveals to Penetration Testers

From PDF and Office Documents:

PDF Metadata Analysis: TargetCo_Q3_2024_Report.pdf

Author: john.smith
Creator: Microsoft Word 2016
Producer: Adobe Acrobat Pro DC 2023.006.20360
CreationDate: 2024-09-15T14:32:11+00:00
ModDate: 2024-09-18T09:15:33+00:00
Title: Q3 2024 Financial Results
Subject: Investor Relations
Keywords: financial, quarterly, results
Company: Target Company Inc.

Template: C:\Users\j.smith\AppData\Roaming\Microsoft\Templates\TC_Corporate_Template.dotx
          ↑ Internal Windows username revealed: j.smith
          ↑ Internal path structure revealed: C:\Users\j.smith\AppData...
          ↑ Template server path may reveal file server naming convention

Last Modified By: Jane Doe
Previous Author: Robert Johnson

Comments in document: 
[Internal Note - TO BE REMOVED]: The Q3 numbers exclude the data breach costs pending legal review

Intelligence extracted:

Username: j.smith → likely email j.smith@targetco.com
Internal path reveals Windows workstation structure
Software versions: Word 2016 → potentially vulnerable to specific exploits
Adobe Acrobat version: Specific version can be checked against CVE database
Document history reveals additional employee names (Robert Johnson, Jane Doe)
The comment "TO BE REMOVED" reveals sensitive business information

From JPEG/Image EXIF Data:

# Extract EXIF from an image
exiftool office_photo.jpg

# Output:
File Name                      : office_photo.jpg
File Size                      : 4.2 MB
File Modification Date/Time    : 2024:03:15 14:30:22
Make                           : Apple
Camera Model Name              : iPhone 14 Pro
Software                       : 17.2.1
Date/Time Original             : 2024:03:15 09:15:44
GPS Latitude                   : 37° 47' 22.40" N    ← EXACT LOCATION
GPS Longitude                  : 122° 25' 10.20" W   ← EXACT LOCATION
GPS Altitude                   : 45.3 m Above Sea Level
GPS Speed                      : 0 km/h (stationary)

GPS coordinates from a photo taken inside an office building reveal:

Precise physical address of the office
Floor-level precision (altitude data)
Confirmation of office location for physical penetration testing planning

Metadata Extraction Tools

FOCA (Fingerprinting Organizations with Collected Archives)

FOCA is specifically designed for extracting and analyzing metadata from documents found during penetration testing reconnaissance. It:

Automatically finds documents published on a target website
Downloads documents from Google, Bing, and DuckDuckGo
Extracts metadata from all found documents
Aggregates usernames, software versions, email addresses, and internal paths
Builds a visual map of internal server names and usernames

Website: https://github.com/ElevenPaths/FOCA (Windows)

Process:

Enter target domain
FOCA searches Google/Bing for documents (.pdf, .docx, .xlsx, .pptx, .txt)
Downloads all found documents
Extracts metadata from each document
Displays aggregated intelligence: usernames, software, printers, servers

ExifTool

The most comprehensive metadata extraction tool — supports 200+ file formats.

# Install
sudo apt install libimage-exiftool-perl
# or
brew install exiftool

# Basic extraction
exiftool document.pdf

# Recursive extraction (all files in directory)
exiftool -r /path/to/documents/

# Extract specific fields
exiftool -Author -Creator -Software document.pdf

# Extract GPS from image and convert to decimal degrees
exiftool -GPSLatitude -GPSLongitude -GPSAltitude -n photo.jpg

# Output as JSON
exiftool -json document.pdf

# Bulk process and output CSV
exiftool -csv *.pdf > metadata_results.csv

# Remove all metadata (defensive use)
exiftool -All= -overwrite_original document.pdf

metagoofil

Searches Google for target domain documents and extracts metadata:

# Install
pip3 install metagoofil

# Basic usage
metagoofil -d targetco.com -t pdf,doc,xls,ppt,odp,ods,docx,xlsx,pptx -l 100 -o /tmp/metagoofil_output

# Options:
# -d: target domain
# -t: file types to search
# -l: limit to N results per file type
# -o: output directory

mat2 (Metadata Anonymisation Toolkit)

# Install
pip3 install mat2

# Extract metadata
mat2 --show document.pdf

# Remove metadata (defensive)
mat2 --inplace document.pdf

The Metadata Intelligence Workflow

Professional metadata reconnaissance process:

#!/bin/bash
# Automated document metadata extraction

DOMAIN="targetco.com"
OUTPUT_DIR="/tmp/metadata_recon"
mkdir -p $OUTPUT_DIR/docs $OUTPUT_DIR/images

# Step 1: Download documents from target website
wget --recursive --level=2 --accept "*.pdf,*.docx,*.xlsx,*.pptx,*.doc,*.xls,*.ppt" \
     --directory-prefix=$OUTPUT_DIR/docs \
     "https://$DOMAIN"

# Also search Google for cached documents (use metagoofil)
metagoofil -d $DOMAIN -t pdf,docx,xlsx,pptx -l 50 -o $OUTPUT_DIR/docs

# Step 2: Extract metadata from all found documents
exiftool -csv -r $OUTPUT_DIR/docs > $OUTPUT_DIR/all_metadata.csv

# Step 3: Extract unique usernames
grep -i "author\|creator\|last.modified" $OUTPUT_DIR/all_metadata.csv | \
awk -F',' '{print $2}' | sort -u > $OUTPUT_DIR/usernames.txt

# Step 4: Extract software versions
grep -i "producer\|creator.tool\|software" $OUTPUT_DIR/all_metadata.csv | \
awk -F',' '{print $2}' | sort -u > $OUTPUT_DIR/software_versions.txt

echo "Metadata extraction complete."
echo "Usernames found: $(wc -l < $OUTPUT_DIR/usernames.txt)"
echo "Software versions: $(wc -l < $OUTPUT_DIR/software_versions.txt)"

3.1.17 Web Archiving, Caching, and Public Code Repositories

The Wayback Machine — Internet Archive

Website: https://web.archive.org/

Operated by: Internet Archive (non-profit)

Coverage: Over 800 billion web pages archived since 1996

The Wayback Machine stores historical snapshots of websites and is one of the most powerful passive reconnaissance resources because it reveals:

1. Content no longer on the live site:

Removed pages that contained sensitive information
Old employee directories (with names, phone numbers, emails)
Discontinued products or services
Former technology partners (revealing integrations)
Old contact pages with direct employee emails
Outdated documentation revealing architecture

2. Historical technology stack:

Old robots.txt files revealing admin paths
Former CMS or platform (may still be running on subdomains)
Source code comments with internal references
JavaScript files revealing API endpoints

3. Behavioral analysis:

How frequently the site was updated
When major technology migrations occurred
Timeline of organizational changes

Using the Wayback Machine:

# Wayback Machine API
# Get all snapshots for a URL
curl "https://archive.org/wayback/available?url=targetco.com"

# Get all saved URLs for a domain (CDX API)
curl "https://web.archive.org/cdx/search/cdx?url=*.targetco.com/*&output=text&fl=original&collapse=urlkey&limit=10000" \
> wayback_urls.txt

# Filter for interesting file types
grep -E "\.(php|asp|aspx|jsp|py|rb|txt|bak|sql|env|config|log)$" wayback_urls.txt

# Tool: waybackurls
waybackurls targetco.com | tee /tmp/wayback_urls.txt

# Tool: gau (getallurls) — combines Wayback, CommonCrawl, and VirusTotal
gau targetco.com | tee /tmp/gau_urls.txt

# Look for interesting paths in historical URLs
cat /tmp/wayback_urls.txt | grep -E "(admin|config|backup|test|dev|api|internal|login|password|credential)"

High-Value Wayback Machine Findings:

Historical URLs discovered for targetco.com:
https://targetco.com/admin/phpMyAdmin/  ← Database admin interface (removed from live site)
https://targetco.com/backup/            ← Backup directory (no longer live)
https://targetco.com/wp-admin/          ← WordPress admin (site no longer uses WP but old files may exist)
https://targetco.com/test/              ← Test environment once publicly accessible
https://targetco.com/api/v1/            ← Old API version (may still be active but undocumented)

These historical paths should be tested against the live site — web servers frequently retain files and directories even when they are removed from the primary navigation.

Google Cache

Google maintains cached copies of indexed web pages. While less comprehensive than the Wayback Machine, Google Cache is more recent.

# Access Google's cached version of a page
cache:targetco.com/employees
cache:www.targetco.com/contact

# Search for cached version of a specific page
# site:google.com/search?q=cache:targetco.com/page

Common Crawl

Common Crawl maintains petabyte-scale archives of web pages and makes them freely available:

# Search Common Crawl index
curl "https://index.commoncrawl.org/CC-MAIN-2024-04-index?url=*.targetco.com/*&output=json" | \
head -100 | python3 -m json.tool

Public Code Repositories — Beyond GitHub

GitHub (Primary)

Largest code repository hosting platform
High probability of finding target-related code
Covered comprehensively in Section 3.1.11

GitLab (Public Instances)

site:gitlab.com "targetco"
site:gitlab.com "targetco.com"

Bitbucket

site:bitbucket.org "targetco"

SourceForge
Older open source projects may be hosted here:

site:sourceforge.net "targetco"

npm (Node Package Manager)

JavaScript developers frequently publish packages to npm that contain internal organizational references:

# Search npm for organization-related packages
curl "https://registry.npmjs.org/-/v1/search?text=targetco&size=50"

# Examine package.json files in found packages for:
# - Internal API URLs
# - Organization names
# - Developer email addresses
# - Internal tool dependencies

PyPI (Python Package Index)

# Search PyPI for organization-related packages
curl "https://pypi.org/pypi/targetco-sdk/json"

Docker Hub

Docker images often contain:

Application configurations
Environment variable templates
Internal tool references
Default credentials (a significant security risk)

# Search Docker Hub for organization images
curl "https://hub.docker.com/v2/search/repositories/?query=targetco"

# Pull and examine layers of a public image (active — use with caution)
docker pull targetco/public-app
docker history targetco/public-app
docker inspect targetco/public-app

robots.txt and sitemap.xml

These files are designed to guide search engine crawlers but inadvertently reveal application structure.

robots.txt:

curl https://targetco.com/robots.txt

# Example output:
User-agent: *
Disallow: /admin/
Disallow: /internal/
Disallow: /api/v2/
Disallow: /staging/
Disallow: /backup/
Disallow: /config/
Disallow: /phpMyAdmin/
Disallow: /wp-admin/
Sitemap: https://www.targetco.com/sitemap.xml

Every Disallow entry is a path the organization does NOT want crawled — which is exactly the list of paths most interesting to a penetration tester.

sitemap.xml:

curl https://targetco.com/sitemap.xml

# May reveal:
# - Complete URL structure of the website
# - API documentation pages
# - Admin functionality pages
# - Application module names

3.1.18 Finding Out About the Organization — Aggregation Techniques

At this stage of passive reconnaissance, the goal is to aggregate all collected intelligence into a coherent, actionable intelligence picture. This synthesis phase is what separates basic reconnaissance from professional intelligence analysis.

Intelligence Aggregation Framework

The Target Intelligence Report Structure:

=== PASSIVE RECONNAISSANCE REPORT: TARGET COMPANY INC. ===
Engagement: [Engagement Reference]
Date: [Date]
Analyst: [Name]
Classification: CONFIDENTIAL — Client Eyes Only

1. ORGANIZATIONAL OVERVIEW
   1.1 Corporate Structure
   1.2 Key Personnel
   1.3 Physical Locations
   1.4 Business Lines and Products

2. TECHNICAL INFRASTRUCTURE
   2.1 Domain Portfolio
   2.2 IP Address Space and ASN
   2.3 External Facing Services
   2.4 Cloud Services Identified
   2.5 Technology Stack

3. EMAIL AND COMMUNICATION INFRASTRUCTURE
   3.1 Email Provider
   3.2 Email Security Posture (SPF/DKIM/DMARC)
   3.3 Confirmed Email Addresses

4. PERSONNEL INTELLIGENCE
   4.1 Key Technical Personnel
   4.2 Security Team Members
   4.3 Email Format
   4.4 Social Engineering Attack Vectors

5. EXPOSURE INTELLIGENCE
   5.1 Breach History
   5.2 Leaked Credentials
   5.3 Reputation Assessment
   5.4 Dark Web Presence

6. HISTORICAL INTELLIGENCE
   6.1 Legacy Systems and Paths
   6.2 Technology Transitions
   6.3 Historical IP Addresses

7. VULNERABILITY INDICATORS
   7.1 SSL/TLS Weaknesses
   7.2 Subdomain Takeover Candidates
   7.3 Shadow IT Identified
   7.4 Unmanaged Assets

8. ATTACK SURFACE MAP
   8.1 Priority Targets (High)
   8.2 Priority Targets (Medium)
   8.3 Social Engineering Targets

Organizational Intelligence Sources

Annual Reports and Investor Relations:

Public companies publish annual reports (10-K in the US) that contain:

Detailed business unit descriptions
Risk factors (explicitly naming IT dependencies and security risks)
Employee count by region
Subsidiary and acquisition information
IT spending disclosures
Named executive officers

# SEC EDGAR for US public company filings
curl "https://efts.sec.gov/LATEST/search-index?q=%22Target+Company%22&dateRange=custom&startdt=2023-01-01&enddt=2024-12-31&forms=10-K"

Press Releases and News:

# Search news archives
site:prnewswire.com "Target Company"
site:businesswire.com "Target Company"
site:globenewswire.com "Target Company"

# Google News
site:news.google.com "Target Company" technology

Conference Presentations and Academic Papers:

Employees frequently present at conferences (RSA, DEF CON, Black Hat, AWS re:Invent) revealing:

Specific technologies and architectures
Security challenges and solutions implemented
Vendor relationships

# Search for conference presentations
site:slideshare.net "Target Company" 2023
site:speakerdeck.com "Target Company"
"Target Company" site:youtube.com conference presentation 2024

3.1.19 Advanced Searches — Google Dorking and Beyond

Google Dorking (also known as Google Hacking) uses advanced Google search operators to find information that is technically public but not easily discovered through normal search.

Google Search Operators — Complete Reference

Operator	Syntax	Example	Use Case
`site:`	`site:domain.com`	`site:targetco.com filetype:pdf`	Search within specific domain
`filetype:`	`filetype:ext`	`site:targetco.com filetype:xlsx`	Find specific file types
`intitle:`	`intitle:keyword`	`intitle:"index of" site:targetco.com`	Pages with keyword in title
`inurl:`	`inurl:keyword`	`inurl:admin site:targetco.com`	Pages with keyword in URL
`intext:`	`intext:keyword`	`intext:"confidential" site:targetco.com`	Pages with keyword in body
`cache:`	`cache:url`	`cache:targetco.com/employees`	Google's cached version
`link:`	`link:url`	`link:targetco.com`	Pages linking to target
`related:`	`related:url`	`related:targetco.com`	Websites similar to target
`"..."`	`"exact phrase"`	`"Target Company" "database password"`	Exact phrase match
`-`	`-keyword`	`site:targetco.com -www`	Exclude keyword
`*`	`*`	`"targetco * password"`	Wildcard
`OR` / `\	`	`term1 OR term2`	`"targetco.com" password OR credential`
`AND`	`term1 AND term2`	`site:targetco.com AND intext:password`	Boolean AND
`before:`	`before:YYYY-MM-DD`	`site:targetco.com before:2020-01-01`	Results before date
`after:`	`after:YYYY-MM-DD`	`site:targetco.com after:2023-01-01`	Results after date
`numrange:`	`numrange:N-M`	`targetco.com numrange:1-65535`	Numeric range

The Google Hacking Database (GHDB)

The GHDB, maintained by Exploit-DB, contains thousands of tested Google dorks organized by category. Every serious penetration tester references the GHDB regularly.

Website: https://www.exploit-db.com/google-hacking-database

GHDB Categories:

Footholds (login portals, admin interfaces)
Files Containing Usernames
Sensitive Directories
Web Server Detection
Vulnerable Files
Vulnerable Servers
Error Messages
Files Containing Juicy Info
Files Containing Passwords
Sensitive Online Shopping Info
Network or Vulnerability Data
Pages Containing Login Portals
Various Online Devices

High-Value Google Dorks for Penetration Testing

Finding Login Portals:

site:targetco.com intitle:"login"
site:targetco.com intitle:"sign in"
site:targetco.com inurl:"/login"
site:targetco.com inurl:"/admin"
site:targetco.com inurl:"/portal"
site:targetco.com inurl:"/wp-admin"
site:targetco.com inurl:"/cpanel"
site:targetco.com inurl:"/webmail"
site:targetco.com inurl:"/owa"   # Outlook Web Access
site:targetco.com inurl:"/citrix"
site:targetco.com inurl:"/vpn"
site:targetco.com inurl:"/remote"
site:targetco.com inurl:"/sslvpn"

Finding Exposed Files:

site:targetco.com filetype:pdf
site:targetco.com filetype:docx
site:targetco.com filetype:xlsx "confidential"
site:targetco.com filetype:txt "password"
site:targetco.com filetype:log
site:targetco.com filetype:bak   # Backup files
site:targetco.com filetype:sql   # Database dumps
site:targetco.com filetype:conf  # Configuration files
site:targetco.com filetype:config
site:targetco.com filetype:env   # Environment files
site:targetco.com filetype:xml inurl:config
site:targetco.com filetype:yml   # YAML config files
site:targetco.com filetype:json  # JSON config files (often API keys)
site:targetco.com filetype:ini   # Windows config files
site:targetco.com filetype:php inurl:config

Finding Sensitive Information:

site:targetco.com "index of /"
site:targetco.com "parent directory"
site:targetco.com intitle:"index of" passwd
site:targetco.com intitle:"index of" ".htpasswd"
site:targetco.com "powered by" inurl:admin
site:targetco.com intext:"sql syntax near"   # SQL error messages
site:targetco.com intext:"MySQL server version"  # MySQL errors
site:targetco.com intext:"Warning: mysql_fetch_array()"  # PHP MySQL errors
site:targetco.com intext:"ORA-00933"   # Oracle errors
site:targetco.com intext:"Traceback (most recent call last)"  # Python errors
site:targetco.com "Exception Details" "Stack Trace"  # .NET exceptions
site:targetco.com "JDBC" "SQLException"   # Java database errors

Finding Exposed Credentials:

site:targetco.com "password" filetype:txt
site:targetco.com "pwd=" filetype:txt
"@targetco.com" "password"   # Searching all of web for org email + password
"@targetco.com" filetype:xls "password"  # Spreadsheets with passwords
site:github.com "targetco.com" "password"
site:github.com "targetco.com" "api_key"
site:github.com "targetco.com" "secret"
site:pastebin.com "targetco.com"

Finding Network Equipment and Infrastructure:

site:targetco.com intitle:"router"
site:targetco.com intitle:"cisco" "login"
site:targetco.com intitle:"FortiGate"
site:targetco.com intitle:"NetScreen"
site:targetco.com intitle:"SonicWALL"
site:targetco.com intitle:"pfSense"
site:targetco.com "Cisco Systems" intitle:"login"

Finding Exposed Development Environments:

site:targetco.com inurl:dev
site:targetco.com inurl:staging
site:targetco.com inurl:test
site:targetco.com inurl:qa
site:targetco.com inurl:uat   # User Acceptance Testing
site:targetco.com inurl:demo
site:targetco.com inurl:beta

Finding Remote Access:

site:targetco.com inurl:"/remote" intitle:"Remote Desktop"
site:targetco.com intitle:"Citrix Gateway"
site:targetco.com intitle:"SSL VPN"
site:targetco.com inurl:"/RDWeb"  # Remote Desktop Web Access
site:targetco.com intitle:"VMware Horizon"  # VMware VDI

Beyond Google — Other Search Engine Dorking

Bing Dorks:

Bing sometimes indexes content that Google does not, particularly from newer or smaller sites.

site:targetco.com filetype:pdf
ip:203.0.113.50   # Bing: all pages on this IP (no Google equivalent)

DuckDuckGo:

site:targetco.com inurl:admin
# DuckDuckGo also has a Bing-powered index with slightly different coverage

Yandex (for Russian and Eastern European targets):

site:targetco.ru
host:targetco.ru

3.1.20 Open-Source Intelligence (OSINT) Gathering — Frameworks and Automation

This section consolidates the methodology, frameworks, and advanced automation tools used by senior penetration testers and intelligence analysts for comprehensive OSINT campaigns.

The OSINT Framework in Practice

Website: https://osintframework.com/

The OSINT Framework is not just a list of tools — it is a structured intelligence methodology translated into an interactive reference. Professional use of OSINT Framework involves:

Step 1: Identify starting data points

Organization name
Domain name
IP address
Email address
Employee names
Phone numbers

Step 2: Navigate to appropriate branch
For each data point, navigate to the corresponding OSINT Framework branch and systematically execute relevant tools.

Step 3: Document and pivot
Record all findings. Each new piece of intelligence becomes a new starting point for additional queries.

Complete OSINT Framework Branches Relevant to Penetration Testing:

OSINT Framework
├── Domain Name
│   ├── WHOIS Records → DomainTools, ViewDNS
│   ├── Subdomains → crt.sh, Subfinder, Amass
│   ├── DNS Records → SecurityTrails, DNSdumpster
│   ├── Website → WaybackMachine, BuiltWith, Wappalyzer
│   └── Email Addresses → Hunter.io, theHarvester
│
├── Email Address
│   ├── Email Reputation → MXToolbox
│   ├── Breach Data → HaveIBeenPwned, Dehashed
│   ├── Email Headers → MXHeaders
│   └── Social Networks linked to email
│
├── IP Address
│   ├── Geolocation → MaxMind, ip-api.com
│   ├── Hosting Provider → ARIN WHOIS
│   ├── Open Ports → Shodan, Censys
│   ├── Reverse DNS → ViewDNS
│   └── Blacklists → MXToolbox Blacklist Check
│
├── Username
│   ├── Social Networks → Sherlock, WhatsMyName
│   ├── Forum Search
│   └── Gaming Profiles
│
├── Person
│   ├── Public Records → Spokeo, Intelius
│   ├── Social Networks → LinkedIn, Twitter, Facebook
│   ├── Email Search → Hunter.io
│   ├── Photo Search → Google Images
│   └── Phone Numbers → Twilio Lookup
│
└── Organization
    ├── Official Records → LinkedIn Company Page, SEC EDGAR
    ├── Job Listings → LinkedIn Jobs, Indeed, Glassdoor
    ├── Financials → OpenCorporates, Companies House
    ├── Employees → LinkedIn Employee Search
    └── Technology → BuiltWith, Wappalyzer, job listings

OSINT Combine Advanced Workflows

Website: https://www.osintcombine.com/

Key OSINT Combine tools for professional use:

1. Social Media Search (Cross-Platform Username Investigation)

When a username or email is discovered, OSINT Combine provides an efficient cross-platform search capability without requiring separate manual searches on each platform.

2. Reverse Image Search

Critical for verifying identities and finding additional accounts associated with a face:

Confirm that a LinkedIn profile photo matches other profiles (verifying identity)
Find additional social media accounts using the same profile photo
Detect fake LinkedIn profiles (using stock photos)

3. Map Searching — Geolocation OSINT

Tools for extracting and verifying geographic information from images and social media posts:

Extract GPS coordinates from image EXIF data
Cross-reference GPS coordinates with satellite imagery
Verify office building locations for physical penetration testing

4. Domain Investigation Batch Processing

When multiple domains are discovered (from CT logs, WHOIS, job listings), OSINT Combine enables bulk processing.

SMART — Advanced Usage

Website: https://smart.myosint.training/

Professional SMART workflows:

When encountering an unusual intelligence requirement (e.g., "find corporate registration records for a subsidiary in Singapore"), SMART quickly surfaces specialized databases and tools that would otherwise require extensive research:

SMART Search: "corporate registry singapore"
Results: 
- ACRA (Accounting and Corporate Regulatory Authority of Singapore)
- Singapore Business Database
- ASEAN OSINT resources
- Regional investigative journalism databases

This makes SMART particularly valuable for:

Country-specific reconnaissance
Specialized industry databases (maritime, aviation, telecommunications)
Non-English language resources

SpiderFoot HX — Cloud-Automated OSINT

Website: https://www.spiderfoot.net/hx/

SpiderFoot HX is the commercial, cloud-hosted version of SpiderFoot. Advantages over self-hosted:

No infrastructure to manage
Pre-configured API keys for major data sources
Team collaboration features
Automated scheduled scans for continuous monitoring
Historical scan comparison (track changes in target's attack surface)

Use case: Many professional penetration testing firms run SpiderFoot HX as a continuous monitoring service for clients between annual penetration tests, alerting on new assets, subdomain additions, and credential exposures.

Building Automated OSINT Pipelines

Senior penetration testers build automated pipelines that chain multiple tools together:

#!/bin/bash
# Professional Passive OSINT Pipeline
# Run from Kali Linux with all tools installed

DOMAIN="targetco.com"
ORG="Target Company Inc."
OUTPUT="/tmp/osint_${DOMAIN}_$(date +%Y%m%d)"
mkdir -p $OUTPUT

echo "[Phase 1] DNS Intelligence"
# Subdomains from Certificate Transparency
subfinder -d $DOMAIN -all -recursive -o $OUTPUT/subdomains_subfinder.txt
amass enum -passive -d $DOMAIN -o $OUTPUT/subdomains_amass.txt

# DNS records
dig $DOMAIN ANY +short > $OUTPUT/dns_any.txt
dig $DOMAIN MX +short > $OUTPUT/dns_mx.txt
dig $DOMAIN TXT +short > $OUTPUT/dns_txt.txt
dig _dmarc.$DOMAIN TXT +short > $OUTPUT/dmarc.txt

echo "[Phase 2] Historical Intelligence"
# Wayback Machine URLs
waybackurls $DOMAIN > $OUTPUT/wayback_urls.txt
gau $DOMAIN > $OUTPUT/gau_urls.txt

echo "[Phase 3] Email and Personnel"
# Email harvesting
theHarvester -d $DOMAIN -b all -l 500 -f $OUTPUT/theharvester.html

# Subdomain resolution
cat $OUTPUT/subdomains_subfinder.txt $OUTPUT/subdomains_amass.txt | sort -u > $OUTPUT/all_subdomains.txt
dnsx -l $OUTPUT/all_subdomains.txt -o $OUTPUT/resolved_subdomains.txt

echo "[Phase 4] SSL Certificate Analysis"
# Certificate transparency mining
curl "https://crt.sh/?q=%.${DOMAIN}&output=json" 2>/dev/null | \
python3 -c "import json,sys; [print(e.get('name_value','')) for e in json.load(sys.stdin)]" | \
sort -u > $OUTPUT/ct_subdomains.txt

echo "[Phase 5] OSINT Aggregation"
# Combine all discovered subdomains
cat $OUTPUT/subdomains_subfinder.txt \
    $OUTPUT/subdomains_amass.txt \
    $OUTPUT/ct_subdomains.txt | \
    sort -u > $OUTPUT/MASTER_subdomains.txt

echo "Pipeline complete. Results: $OUTPUT"
echo "Total subdomains discovered: $(wc -l < $OUTPUT/MASTER_subdomains.txt)"

3.1.21 Shodan — The Search Engine for Everything Connected

Website: https://www.shodan.io/

Type: Internet-connected device search engine

Cost: Free (limited), Freelancer ($59/month), Small Business ($299/month), Corporate (custom)

Created by: John Matherly (2009)

Shodan is one of the most powerful passive reconnaissance tools in existence. Unlike Google, which indexes website content, Shodan continuously scans the entire Internet and indexes the service banners — the metadata returned by servers when a connection is made.

This means Shodan knows:

Every IP address with an open port on the Internet
What software is running on each port (with version numbers)
SSL certificate details
Geographic location and ISP of every device
Historical data going back years

What Shodan Indexes

Shodan does not wait for you to search — it continuously probes every IP address on the Internet across hundreds of ports and stores what each service returns:

Shodan Banner Example for a Web Server:
HTTP/1.1 200 OK
Server: Apache/2.4.41 (Ubuntu)
X-Powered-By: PHP/7.4.3
Content-Type: text/html
Date: Sun, 01 Jan 2024 12:00:00 GMT

This banner tells Shodan (and therefore any searcher):

Operating system: Ubuntu
Web server: Apache 2.4.41 (specific vulnerability-checkable version)
PHP version: 7.4.3 (end-of-life — no security patches)

Shodan indexing covers:

HTTP (80, 8080, 8443, 8888, etc.)
HTTPS (443)
SSH (22)
FTP (21)
Telnet (23)
SMB (445)
RDP (3389)
VNC (5900-5901)
SNMP (161)
DNS (53)
SMTP/IMAP/POP3 (25, 143, 110)
Database ports (3306 MySQL, 5432 PostgreSQL, 1433 MSSQL, 27017 MongoDB)
Industrial control systems (Modbus 502, DNP3 20000, EtherNet/IP 44818)
IoT devices (cameras, printers, routers, NAS)
Kubernetes API (6443, 8001)
Docker API (2375, 2376)
Elasticsearch (9200, 9300)
Redis (6379)
memcached (11211)
Cassandra (9042)

Shodan Search Syntax

Basic Searches:

# Search for services belonging to an organization
org:"Target Company Inc."

# Search for services on a specific IP
ip:203.0.113.50

# Search for services in an IP range
net:203.0.113.0/24

# Search for services on a specific hostname
hostname:targetco.com

# Search for services in a country
country:US org:"Target Company"

# Search for specific ports
org:"Target Company" port:3389   # RDP exposed
org:"Target Company" port:22     # SSH exposed
org:"Target Company" port:23     # Telnet (critical finding)
org:"Target Company" port:445    # SMB
org:"Target Company" port:3306   # MySQL

# Search for specific products/technologies
org:"Target Company" product:"Apache httpd"
org:"Target Company" product:"nginx"
org:"Target Company" product:"Microsoft IIS"
org:"Target Company" product:"OpenSSH"
org:"Target Company" product:"MySQL"

Advanced Filters:

# Find specific software versions
org:"Target Company" product:"Apache httpd" version:"2.4.41"

# Find SSL certificate information
ssl.cert.subject.CN:"targetco.com"
ssl.cert.subject.O:"Target Company Inc."

# Find by SSL certificate expiry (expired certs)
ssl.cert.expired:true org:"Target Company"

# Find devices with specific vulnerabilities (Shodan CVE search)
org:"Target Company" vuln:CVE-2021-44228   # Log4Shell
org:"Target Company" vuln:CVE-2021-26855   # ProxyLogon (Exchange)
org:"Target Company" vuln:CVE-2019-19781   # Citrix Netscaler

# Find industrial control systems
org:"Target Company" port:502   # Modbus
org:"Target Company" port:44818  # EtherNet/IP (Rockwell PLC)

# Find default credentials still in use (content in banner)
org:"Target Company" "default password"
org:"Target Company" http.title:"admin" http.status:200

# Find specific web technologies
org:"Target Company" http.component:"WordPress"
org:"Target Company" http.component:"Joomla"
org:"Target Company" http.component:"Drupal"
org:"Target Company" http.component:"Tomcat"

# Find cloud storage open to public
org:"Target Company" http.title:"Index of /"

Shodan Dorks (High-Value Search Patterns):

# Exposed databases
org:"Target Company" product:"MongoDB" -authentication
org:"Target Company" port:9200 product:"Elastic"
org:"Target Company" port:6379 product:"Redis"
org:"Target Company" port:27017

# Remote desktop and management
org:"Target Company" port:3389 product:"Remote Desktop Protocol"
org:"Target Company" port:5900 product:"VNC"
org:"Target Company" has_screenshot:true port:3389   # Screenshots of exposed RDP logins

# Printers
org:"Target Company" port:9100 product:"Jetdirect"
org:"Target Company" http.title:"Printer"

# Network devices
org:"Target Company" port:161 product:"SNMP"   # SNMP (community string exposure)
org:"Target Company" http.title:"Cisco" port:443

# VPN gateways
org:"Target Company" product:"Fortinet SSL VPN"
org:"Target Company" product:"Palo Alto Networks"
org:"Target Company" product:"Pulse Secure"
org:"Target Company" product:"Citrix"

# Building management and IoT
org:"Target Company" product:"BACnet"
org:"Target Company" port:47808

Shodan CLI Tool

# Install Shodan CLI
pip3 install shodan

# Initialize with API key
shodan init YOUR_API_KEY

# Basic search
shodan search 'org:"Target Company Inc."'

# Show all services for an IP
shodan host 203.0.113.50

# Count results
shodan count 'org:"Target Company Inc."'

# Download results
shodan download --limit 1000 targetco_results 'org:"Target Company Inc."'
shodan parse --fields ip_str,port,transport,product,version targetco_results.json.gz

# Alert on new services (monitoring)
shodan alert create "Target Company Monitor" --ip 203.0.113.0/24

# Show alerts
shodan alert list

Shodan for Vulnerability Prioritization

One of Shodan's most powerful features is the CVE Search — identifying known vulnerabilities in discovered systems:

# Search for Log4Shell vulnerable systems (CVE-2021-44228)
shodan search 'org:"Target Company" vuln:CVE-2021-44228'

# Example result showing organization's vulnerable systems
203.0.113.50    80/tcp   Apache Tomcat 9.0.37 [CVE-2021-44228]
203.0.113.51    8080/tcp Apache Tomcat 8.5.51 [CVE-2021-44228]

This is intelligence that goes directly into the exploitation phase — during active testing, these systems are immediate high-priority targets.

Shodan's Screenshotting Feature

Shodan captures screenshots of visual services (RDP, VNC, web interfaces) when they are discovered. This allows passive viewing of:

Windows RDP login screens (revealing Windows version and computer name)
VNC remote desktop sessions (occasionally displaying active sessions)
Web-based control panels and administrative interfaces
Industrial HMI (Human-Machine Interface) screens

# Find systems with screenshots
shodan search 'org:"Target Company" has_screenshot:true'

# View screenshots in Shodan web interface
# https://www.shodan.io/search?query=org%3A%22Target+Company%22+has_screenshot%3Atrue

Censys — Shodan Alternative

Censys is a comparable tool to Shodan with some distinct advantages:

# Censys CLI
pip3 install censys

# Search hosts
censys search 'organization.name="Target Company Inc."' --index-type HOSTS

# Search certificates
censys search 'parsed.subject.organization="Target Company Inc."' --index-type CERTS

# Detailed host information
censys view HOSTS 203.0.113.50

Shodan vs. Censys comparison:

Feature	Shodan	Censys
Scan frequency	Continuous	Continuous
Protocol coverage	Very broad	Broad
Historical data	Yes	Yes
Certificate search	Yes	Excellent (dedicated index)
Free tier	Yes (limited)	Yes (limited)
Vulnerability data	CVE tagging	CVE tagging
Screenshot capture	Yes	Limited
IoT/OT focus	Strong	Moderate
ASN search	Yes	Yes

3.1.22 Breach Data Intelligence — Leaked Credentials and Exposure Monitoring

Breach intelligence is one of the most immediately actionable categories of passive reconnaissance data. Discovering that an organization's employees have passwords in public breach databases provides a direct credential stuffing attack vector.

Understanding the Breach Data Ecosystem

Every major data breach eventually results in the stolen data being traded, sold, and ultimately made public in various forums, paste sites, and dedicated breach databases. The timeline typically follows:

Breach Occurs
    ↓ (Days to months)
Data sold on dark web markets (private, expensive)
    ↓ (Months to years)
Data traded in hacker forums (semi-private)
    ↓ (Months to years)
Data aggregated and indexed (public, searchable)
    ↓ (Current state)
Data accessible via breach intelligence services

Have I Been Pwned (HIBP)

Website: https://haveibeenpwned.com/

Created by: Troy Hunt (Microsoft Regional Director, security researcher)

Data: 12+ billion records from 600+ breaches

HIBP is the most respected and widely used breach notification service. It provides:

For email addresses:

Which breaches an email address appears in
What data was exposed (password, phone number, physical address, etc.)
Paste sites where the email appears

For domains (enterprise use):

Domain-level search: all compromised email addresses @targetco.com
API access for bulk lookups

# HIBP API for domain search
curl "https://haveibeenpwned.com/api/v3/breacheddomain/targetco.com" \
-H "hibp-api-key: YOUR_API_KEY" \
-H "user-agent: MyPentestApp"

# Check single email
curl "https://haveibeenpwned.com/api/v3/breachedaccount/j.smith@targetco.com" \
-H "hibp-api-key: YOUR_API_KEY" \
-H "user-agent: MyPentestApp"

# Response
[
  {
    "Name": "LinkedIn",
    "BreachDate": "2012-05-05",
    "Description": "LinkedIn breach of 2012 affecting 164 million accounts",
    "DataClasses": ["Email addresses", "Passwords"]
  },
  {
    "Name": "Adobe",
    "BreachDate": "2013-10-04",
    "Description": "Adobe breach affecting 152 million accounts",
    "DataClasses": ["Email addresses", "Password hints", "Passwords", "Usernames"]
  }
]

Intelligence derived from HIBP results:

If 40% of Target Company's employees have been in breaches where passwords were exposed, this reveals:

High likelihood of password reuse across corporate and personal accounts
Historical password patterns (enabling targeted password guessing)
Specific employee email addresses are confirmed active
Breach recency (2012 LinkedIn breach → 12-year-old password likely still reused)

F-Secure Identity Checker

Website: https://www.f-secure.com/en/home/free-tools/identity-checker

F-Secure provides a consumer-facing breach checking tool that uses their own breach intelligence database. As a cybersecurity company, F-Secure has access to breach data not in HIBP.

Professional relevance:

Use alongside HIBP for broader coverage
Can confirm exposure not in HIBP's dataset

HackNotice

Website: https://hacknotice.com/

Type: Breach monitoring and notification service

HackNotice provides breach intelligence with a focus on:

Dark web monitoring for organization data
Real-time breach alerts
Industry-specific breach tracking
Employee exposure assessment

Professional use: HackNotice is typically used as part of a threat intelligence subscription for continuous monitoring rather than point-in-time penetration test reconnaissance.

BreachDirectory

Website: https://breachdirectory.com/

Type: Breach data search with partial password hash exposure

BreachDirectory provides search capability across breach data and, notably, can show partial password hashes — allowing confirmation that a password exists in breach data without exposing the full hash.

# BreachDirectory API
curl "https://breachdirectory.com/api/?func=auto&term=j.smith@targetco.com" \
-H "x-api-key: YOUR_KEY"

# Response includes partial SHA-1 hash
{
  "success": true,
  "result": [
    {
      "email": "j.smith@targetco.com",
      "sha1": "5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8"
    }
  ]
}

The SHA-1 hash can then be looked up in hash databases to recover the plaintext password.

Keeper Security

Website: https://keepersecurity.com/

Type: Password manager and breach monitoring platform

Keeper Security's enterprise product includes:

BreachWatch: Monitors the dark web for employee credential exposure
Event Reporting: Tracks credential-related security events
Compliance Reporting: Generates compliance reports for breached account remediation

Relevance to penetration testing: Keeper's breach monitoring features inform the penetration tester about the sophistication of the client's credential hygiene program. If the client uses BreachWatch and still has many exposed credentials, it indicates employees are not responding to breach notifications.

WhatBreach

GitHub: https://github.com/Ekultek/WhatBreach

Type: Open-source breach data aggregation tool

Created by: Ekultek

WhatBreach is a command-line tool that checks emails against multiple breach databases simultaneously:

# Install
git clone https://github.com/Ekultek/WhatBreach
cd WhatBreach
pip3 install -r requirements.txt

# Single email check
python3 whatbreach.py -e j.smith@targetco.com

# Multiple emails from file
python3 whatbreach.py -l email_list.txt

# Output
[+] Email: j.smith@targetco.com
[*] Found in 3 breaches:
    - LinkedIn (2012): Email, Password
    - Adobe (2013): Email, Password Hint, Username
    - MyFitnessPal (2018): Email, IP Address, Username

[*] Checking pastes...
[+] Found on 2 paste sites:
    - Pastebin: https://pastebin.com/AbCdEfGh
    - Ghostbin: https://ghostbin.com/paste/XyZaBc

LeakLooker

GitHub: https://github.com/woj-ciech/LeakLooker

Type: Tool for finding exposed databases and files

Platform: Python

LeakLooker searches Shodan and other sources for exposed and potentially breached data sources, including:

Exposed MongoDB databases (no authentication)
Exposed Elasticsearch clusters
Exposed CouchDB instances
Exposed files in cloud storage

# Install
git clone https://github.com/woj-ciech/LeakLooker
cd LeakLooker
pip3 install -r requirements.txt

# Search for exposed databases belonging to target
python3 leaklooker.py -s --target targetco.com

# Search Shodan for exposed databases in target's IP range
python3 leaklooker.py --shodan --query 'net:203.0.113.0/24'

Why LeakLooker matters:

Many organizations have accidentally exposed databases to the public Internet:

MongoDB with no authentication (common misconfiguration)
Elasticsearch with no authentication (extremely common)
Redis with no authentication and no bind restriction
Cassandra with no authentication

Discovering these during passive reconnaissance (via Shodan data) is a critical finding that can be reported as a severe vulnerability even before active testing begins.

Buster

GitHub: https://github.com/sham00n/buster

Type: Email to social profile aggregation tool

Buster takes an email address and finds associated accounts on other platforms:

# Install
git clone https://github.com/sham00n/buster
cd buster
pip3 install -r requirements.txt

# Search for accounts associated with an email
python3 buster.py -e j.smith@targetco.com

# Output
[+] Searching for: j.smith@targetco.com
[*] Results:
    - Gravatar: Profile found (avatar reveals face photo)
    - Github: username jsmith-dev (public repositories found)
    - GitLab: username jsmith (repositories found)
    - Disqus: Comments found (reveals opinions and interests)
    - About.me: Personal page found

Intelligence value:

GitHub username → access to their public repositories → potential code disclosure
Personal website → technology preferences, skills, contact information
Gravatar → profile photo (for identity verification in social engineering)

Scavenger

GitHub: https://github.com/rndinfosecguy/Scavenger

Type: OSINT tool for finding breach data from dark web sources

Scavenger aggregates data from dark web sources, IRC channels, and paste sites to find exposed credentials:

# Install
git clone https://github.com/rndinfosecguy/Scavenger
cd Scavenger
pip3 install -r requirements.txt

# Configure API keys in config.py
# Run search
python3 scavenger.py -s j.smith@targetco.com

PwnDB

GitHub: https://github.com/davidtavarez/pwndb

Type: Tool for querying the PwnDB Tor-hidden service for leaked credentials

PwnDB is a dark web database of leaked credentials. The pwndb tool provides a command-line interface for querying it:

# Install
git clone https://github.com/davidtavarez/pwndb
cd pwndb
pip3 install -r requirements.txt

# Requires Tor running locally
sudo service tor start

# Query for email
python3 pwndb.py --target j.smith@targetco.com

# Query for domain (finds all @targetco.com exposures)
python3 pwndb.py --target targetco.com

Important note: PwnDB queries a Tor-based service and returns plaintext or partially-recovered passwords. This intelligence must be handled in accordance with the engagement's legal agreements and data protection obligations.

Credential Stuffing — The Attack Chain

Once breach data is identified, the attack chain is:

1. Identify breached @targetco.com email addresses (HIBP, pwndb, etc.)
         ↓
2. Obtain associated password hashes from breach databases
         ↓
3. Crack password hashes (offline, using hashcat/John the Ripper)
         ↓
4. Test recovered plaintext passwords against corporate systems:
   - Microsoft 365 login (outlook.office365.com)
   - Corporate VPN login
   - Citrix Gateway
   - Web application login
         ↓
5. Successful authentication = confirmed credential reuse vulnerability
   (Password Spraying: test one password across many accounts to avoid lockouts)

Password Spraying in the authorization context:

Password spraying uses a single common password against many accounts, avoiding account lockout:

# Only perform after authorization is granted (ACTIVE reconnaissance)
# Tools used in active phase:
# - Spray
# - MSOLSpray (Microsoft 365)
# - GoMapEnum
# - TREVORspray
# - CredKing (Lambda-based spray for IP rotation)

Dehashed

Website: https://dehashed.com/

Type: Commercial breach intelligence database with comprehensive credential search

Dehashed is a paid service that provides access to one of the largest breach databases, enabling:

Email to password search
Username to password search
IP address to credential search
Domain-wide breach analysis

# Dehashed API
curl "https://api.dehashed.com/search?query=domain:targetco.com&size=100" \
-H "Authorization: Basic $(echo -n 'email@example.com:API_KEY' | base64)"

The Complete Breach Intelligence Workflow

Step 1: Collect all @targetco.com email addresses
  └── Sources: theHarvester, Hunter.io, LinkedIn (via CrossLinked), 
              HIBP domain search, CT log-derived email formats

Step 2: Check all emails against breach databases
  └── Tools: HIBP API, WhatBreach, pwndb, Dehashed

Step 3: Identify accounts with exposed passwords
  └── Prioritize by:
      - Recency of breach (more recent = less likely changed)
      - Number of breaches (multiple breaches = likely password reuser)
      - Seniority of account holder (admin accounts are highest priority)
      - Role (IT admins, security team, finance are high-value)

Step 4: Attempt to recover plaintext passwords
  └── Hashcat, John the Ripper against recovered hashes
  └── Online lookup services for unsalted MD5/SHA1

Step 5: Prepare credential list for active testing phase
  └── Organize by target (VPN, webmail, application)
  └── Document for inclusion in pre-engagement intelligence report

Module 3 — Section 3.2: Performing Active Reconnaissance

CompTIA PenTest+ / Ethical Hacking Certification Series
Professional Reference Guide — GitHub Edition
Written to build real understanding, not just tool familiarity.

3.2.1 Overview — What Active Reconnaissance Really Means
3.2.2 Nmap Scan Types — The Deep Dive
3.2.3 Practice — Nmap in a Real Engagement Context
3.2.4 Types of Enumeration — Extracting Intelligence from Open Ports
3.2.5 Enumeration via Packet Crafting with Scapy
3.2.6 Lab Reference — Enumeration with Nmap
3.2.7 Packet Inspection and Eavesdropping
3.2.8 Practice — Packet Inspection in a Real Scenario
3.2.9 Packet Crafting with Scapy — Full Professional Reference
3.2.10 Network Sniffing with Wireshark — The Complete Guide

3.2.1 Overview — What Active Reconnaissance Really Means

The Moment Everything Changes

There is a very clear line in a penetration testing engagement. Before that line, everything you do is invisible to the target — you are reading public records, browsing archived websites, looking at job listings. You leave no trace, you generate no alerts, and you are at zero legal risk. That is passive reconnaissance.

The moment you cross into active reconnaissance, everything changes. You send packets. Those packets arrive at real machines. Those machines log the connection. Firewalls potentially block and alert on the traffic. Intrusion detection systems start correlating what they see. The target's defenders, if they are watching, now have evidence that someone is probing their network.

This is why the signed authorization document — the Rules of Engagement — is not just a formality. It is the legal instrument that transforms what would be a criminal offense into legitimate, contracted security work. A penetration tester who begins active reconnaissance before the contract is signed is not an ethical hacker. They are committing an offense under the Computer Fraud and Abuse Act in the United States, the Computer Misuse Act in the United Kingdom, and equivalent laws in virtually every other jurisdiction.

So active reconnaissance starts the moment both parties have signed off and the testing window has opened — not a minute earlier.

What Makes Active Reconnaissance Valuable

Think about what passive reconnaissance gives you. You have learned from LinkedIn that the company uses AWS, has a Kubernetes infrastructure team, and recently posted a job for a "Senior Microsoft 365 Administrator." You found some subdomains via certificate transparency logs. You checked their DNS records and discovered they use Microsoft 365 for email. This is valuable — but it is secondhand information. It tells you what the company claims or appears to have, not what is actually running and accessible right now.

Active reconnaissance answers the question that matters most in a penetration test: what is actually exposed and reachable at this moment?

A company might have decommissioned a server six months ago but still have it in their DNS records because nobody updated the documentation. Active reconnaissance reveals it is gone — the IP does not respond. Conversely, that same company might have a test server that was stood up last week, never documented anywhere publicly, but actively listening on port 8080 with default credentials. Passive reconnaissance will never find it. An active scan of their IP range will.

This is the power of active reconnaissance: it shows you the real attack surface, not the documented or intended one.

The Pixel Paradise Network — Understanding Your Target

In the Protego / Pixel Paradise engagement scenario, the network contains a rich mix of device types:

User endpoints include desktops, laptops, mobile devices, and gaming consoles. From a penetration testing perspective, these are interesting because they run user-side software (browsers, email clients, productivity tools) and often have weaker configurations than servers. They are also the primary target of social engineering and client-side attacks.

Facilities endpoints include surveillance cameras, alarm systems, climate control sensors, lighting systems, and VoIP phones. This category — often called Operational Technology (OT) or IoT in enterprise environments — is enormously significant in modern penetration testing. These devices frequently run embedded operating systems, have no patching mechanism, use default credentials, and communicate over unencrypted protocols. A security camera on the same network segment as a database server is not just a surveillance device — it is a potential pivot point.

Intermediary devices are routers and switches. Compromising a router gives visibility into all traffic flowing through it and often allows traffic manipulation. Switches, particularly managed switches, can be exploited through VLAN hopping and ARP poisoning techniques.

Wireless access points provide Wi-Fi. The security of the wireless authentication mechanism (WPA2-PSK, WPA2-Enterprise, WPA3) determines how difficult it is to gain network access without a wired connection.

Firewalls protect the perimeter and potentially segment internal networks. Understanding how a firewall is configured — what it allows, what it blocks, and whether it performs stateful inspection — fundamentally shapes the attack strategy.

Active reconnaissance maps all of these devices systematically.

The Active Reconnaissance Methodology

Active reconnaissance is not random scanning. Professional practitioners follow a structured methodology that builds intelligence layer by layer:

The first step is host discovery — determining which IP addresses in the target range have live hosts. There is no point scanning 65,535 ports on every address in a /16 network (65,534 IP addresses) when perhaps only 200 of those addresses actually have devices on them. Host discovery narrows the field before the intensive work begins.

The second step is port scanning — for each live host, determining which TCP and UDP ports are in an open, closed, or filtered state. This tells you what services the device is offering to the network.

The third step is service and version detection — determining not just that port 443 is open, but that it is running nginx 1.18.0 on Ubuntu 20.04. The specific version of every service is the direct input for vulnerability research.

The fourth step is OS detection — identifying the operating system of each host, which narrows the attack surface further and informs lateral movement strategy.

The fifth step is enumeration — for each discovered service, extracting detailed, service-specific intelligence. If you found SMB is running, enumeration extracts the list of shares, user accounts, and password policy. If SNMP is running, enumeration extracts the device's full configuration and connected network topology.

Each step feeds the next, progressively building a complete picture of the target environment.

3.2.2 Nmap Scan Types — The Deep Dive

What Nmap Actually Is

Nmap — Network Mapper — is the most widely used network reconnaissance tool in the world. It was created by Gordon Lyon (known online as Fyodor) in 1997 and published in Phrack Magazine Issue 51. Nearly three decades later, it remains the standard tool for network reconnaissance because it does its job extraordinarily well: it determines what is on a network, what those things are, and what they are running.

But to understand Nmap deeply, you first need to understand the protocols it manipulates. You cannot use Nmap intelligently — choosing the right scan type for the right situation, interpreting results correctly, recognizing when a result might be a false positive — without understanding what is actually happening at the network level.

TCP — Transmission Control Protocol

TCP is the protocol that the majority of internet applications depend on. When you load a website, send an email, or connect via SSH, you are using TCP. The reason TCP is so widely used comes down to one word: reliability.

TCP is a connection-oriented protocol. Before any data is exchanged, the two parties (let us call them Client and Server) must establish a connection through a process called the three-way handshake. Think of it like a phone call: you dial (SYN), the other person picks up and says hello (SYN-ACK), and you acknowledge that you can hear them (ACK). Only then does the actual conversation begin.

Each TCP packet carries flags in its header — small bits that indicate what type of packet this is and what the sender wants the receiver to do with it. The six classic flags are:

SYN (Synchronize) — This flag says "I want to start a connection with you." It is the first packet in every new TCP connection.

ACK (Acknowledge) — This flag says "I received your last packet." Almost every TCP packet after the initial SYN carries an ACK, because TCP guarantees delivery by requiring every packet to be acknowledged.

FIN (Finish) — This flag says "I am done sending data and want to close this connection gracefully." Unlike RST, a FIN-based close allows the other side to finish sending whatever it still has to send before the connection terminates.

RST (Reset) — This flag says "Abort this connection immediately." It is an emergency stop. When a server receives a connection request to a port where nothing is listening, it sends RST. When a security device detects a suspicious connection, it may inject RST packets to force the connection closed.

PSH (Push) — This flag tells the receiving side to pass the data up to the application immediately rather than buffering it.

URG (Urgent) — This flag indicates that part of the payload is urgent and should be processed before the rest.

Understanding these flags is the foundation of understanding every Nmap scan type, because what Nmap does at its core is send specific combinations of these flags and interpret what comes back.

UDP — User Datagram Protocol

UDP is TCP's simpler, faster, less reliable sibling. While TCP goes through the three-way handshake and guarantees delivery, UDP just sends packets and does not wait to confirm they arrived. This makes UDP faster and more efficient — which is why real-time applications like video streaming, online gaming, VoIP calls, and DNS queries use UDP. If you are in a video call and a packet gets lost, you do not want the video to pause while the system resends that packet — you just want to move on to the next frame. UDP enables that.

For penetration testing, UDP matters enormously because UDP services are the most frequently overlooked and therefore often the least secured. Administrators focus their patching and monitoring efforts on TCP services. UDP services like SNMP (Simple Network Management Protocol), TFTP (Trivial File Transfer Protocol), and DNS often run with default configurations and weak security.

The Port State System

Before diving into specific scan types, you need to understand what Nmap is actually trying to determine: the state of each port.

A port is like a numbered door on a building. The building is the IP address. Each door (port number, 1 through 65535) can be in one of several states:

An open port means a service is actively listening behind that door. If you knock (send a packet), someone answers. Port 443 being open on a web server means there is an HTTPS service ready to accept connections.

A closed port means the door exists and the building is reachable, but nobody is listening behind that specific door. The building (host) is up, but this particular service is not running. The key behavior: a closed port responds to probes — it sends back a RST packet saying "nothing here."

A filtered port is the most ambiguous state. Nmap sent a probe and received no response — but that does not mean the port is open. It might mean a firewall silently dropped the packet without sending any response. Nmap cannot distinguish between "open service that is not responding to this type of probe" and "firewall dropping packets" without additional evidence.

An unfiltered port is specifically the ACK scan result — the port is accessible, but Nmap cannot determine from an ACK probe alone whether it is open or closed.

The states open|filtered and closed|filtered represent Nmap's honest admission that it cannot determine the exact state with the probes it sent.

The TCP Connect Scan (-sT)

Imagine you are trying to determine whether a shop is open. The simplest approach is to walk up to the door, try the handle, and see if it opens. If it opens, the shop is open. If you get a "Sorry, We're Closed" sign, it is closed. If there is a security guard who stops you before you even reach the door, something is blocking you.

The TCP Connect scan works exactly like this. It uses your operating system's built-in networking capability — specifically the connect() system call — to attempt a full, complete connection to each target port. It does not manipulate raw packets. It just asks the operating system: "Please connect to this IP address, this port number."

The operating system performs the complete three-way handshake. If the handshake succeeds (the server sends SYN-ACK and the connection is established), the port is open. Nmap then immediately closes the connection with a RST and moves to the next port. If the target sends a RST during the handshake attempt, the port is closed. If no response comes after a timeout period, the port is filtered.

Why this matters: Because it uses the OS networking stack rather than raw packets, the TCP Connect scan does not require administrative privileges. Anyone can run it. This makes it useful when you are testing from a machine where you do not have root or administrator access — perhaps you are testing from a standard user account on a company workstation, or from a cloud instance where your user lacks elevated privileges.

The significant drawback is detectability. Because the scan completes full TCP connections, every service that receives a connection logs it. Apache logs it. Microsoft IIS logs it. sshd logs it. The MySQL daemon logs it. If a security analyst reviews server logs after a -sT scan, they will find a clear record of rapid sequential connections from the scanner's IP address.

In a real engagement, you would use the TCP Connect scan when you literally have no choice — when you lack the privileges for a SYN scan. Otherwise, you almost always prefer the SYN scan for its speed and reduced logging footprint.

The SYN Scan (-sS) — Understanding the "Stealth" Concept

The SYN scan is Nmap's default scan type when run with root privileges, and it is the most commonly used scan type in professional penetration testing. To understand why, you need to understand one key insight about how logging works in network services.

Most network services only create a log entry when a connection is successfully established — that is, when the three-way handshake has completed. The thinking is: why log connection attempts that never went anywhere? Only completed connections represent real sessions and real interactions.

The SYN scan exploits this behavior. Instead of completing the three-way handshake, it sends just the initial SYN packet. If the target port is open, the server responds with SYN-ACK — saying "yes, I am ready to connect." But instead of completing the handshake with an ACK, Nmap immediately sends a RST — effectively saying "never mind, abort." The connection is torn down before it is ever fully established.

The result: the target's service never sees a completed connection. In many older and simpler services, no log entry is created. The probe and its result (port open or closed) are determined entirely from the SYN-ACK or RST-ACK response to the initial SYN packet, without the handshake ever completing.

This is why it is called a "half-open" scan — the connection is half-opened but never completed.

The "stealth" label deserves an important caveat, though. Against modern enterprise defenses — a Palo Alto NGFW, a Suricata IDS, a Cisco FirePOWER — a SYN scan is not stealthy at all. These systems inspect every packet at the wire level, not just completed connections. They detect SYN scans within seconds based on the rate and pattern of SYN packets from a single source IP. The "stealth" in "stealth scan" refers to its behavior relative to simple, service-level logging — not to modern security infrastructure.

In practical terms, SYN is preferred over TCP Connect because it is faster (no need to wait for handshake completion) and generates fewer log entries in service logs. Against sophisticated defenses, you need timing manipulation and evasion techniques on top of scan type selection.

Requires root or administrator privileges because crafting raw TCP packets (sending a SYN and then an RST rather than a normal ACK) requires direct access to the network stack at a level that the operating system only grants to privileged users.

The UDP Scan (-sU) — The Most Overlooked and Most Important

UDP scanning is unglamorous, slow, and frustrating. It is also absolutely critical and frequently reveals the most interesting vulnerabilities on a network. There is a reason experienced penetration testers always run a UDP scan even when they are pressed for time.

Here is the fundamental problem: UDP has no handshake. You send a UDP packet to a port. One of three things happens. First, the service running on that port receives the packet and sends a response — in which case you know the port is open. Second, nothing is running on that port, and the operating system sends back an ICMP "Port Unreachable" message — in which case you know the port is closed. Third, you receive nothing at all — which could mean the port is open but the service did not respond to your generic probe, or it could mean a firewall dropped the packet.

This ambiguity is what makes UDP scanning slow. For many UDP ports, Nmap cannot send a generic probe and expect a response — it needs to send a protocol-specific probe. For DNS (port 53), it sends a DNS query. For SNMP (port 161), it sends an SNMP request. For TFTP (port 69), it sends a TFTP request. Without a protocol-appropriate probe, the service will not respond, and the port appears filtered even if it is actually wide open.

On top of this, Linux and many other operating systems rate-limit the generation of ICMP Port Unreachable messages to prevent them from being used in denial-of-service attacks. If you are scanning a Linux host, it might generate at most one ICMP Port Unreachable per second. A full 65,535-port UDP scan against a Linux host will therefore take over 18 hours to complete if you are relying on ICMP responses to determine closed ports.

This is why professional practice is to always limit UDP scans to the ports that matter most. The most valuable UDP services for penetration testing are:

SNMP on port 161 is arguably the most impactful UDP service to discover during active reconnaissance. SNMP (Simple Network Management Protocol) is used to monitor and manage network devices — routers, switches, firewalls, servers, printers — from a central management station. The protocol works by querying a database of management information called the MIB (Management Information Base). When accessed with the right community string (essentially a password), SNMP reveals the complete internal state of a device: every interface and its IP address, the routing table, every connected device it has ever communicated with (the ARP table), the list of running processes, installed software, CPU and memory utilization, and in the case of network devices, the complete configuration.

The catch is that the two default community strings — "public" for read-only access and "private" for read-write access — are still in use across millions of devices worldwide because administrators never changed them. Finding SNMP open with the default "public" community string on a network device is one of the most impactful findings in a penetration test: it reveals the entire network topology instantly.

DNS on port 53 is present on every DNS server. Active DNS enumeration — which we cover in detail in the enumeration sections — can reveal zone information, internal hostnames, and occasionally enable zone transfers that dump the entire DNS database.

NTP on port 123 (Network Time Protocol) runs on virtually every networked device. NTP vulnerabilities are occasionally critical (notably NTP amplification attacks in DDoS contexts) but more relevantly, the NTP monlist command on older implementations can dump a list of the last 600 hosts that synchronized time with the server — effectively a list of active network hosts.

TFTP on port 69 (Trivial File Transfer Protocol) is used for network device configuration backups and OS image transfers. It has no authentication mechanism whatsoever. Finding a TFTP server during a penetration test is almost always a critical finding — TFTP servers frequently contain configuration backups of routers and switches, including their complete configurations with password hashes.

The FIN Scan (-sF), NULL Scan (-sN), and Xmas Scan (-sX) — RFC Exploitation

These three scan types belong to a family sometimes called "stealth scans" or "flag manipulation scans." To understand why they exist and what they do, you need to understand RFC 793 — the original specification for TCP published in 1981.

RFC 793 defines how a TCP implementation should respond when it receives a packet that does not match any existing connection. The rule is:

If a port is closed and receives any packet that does not have the RST flag set, the receiving system should send back a RST packet.

If a port is open and receives a packet that does not have the SYN flag set (meaning it is not a legitimate connection initiation), the packet should simply be discarded — ignored entirely. The rationale is that a packet arriving at an open port without being part of a valid connection sequence is malformed or out of order, and the correct behavior is to ignore it.

The FIN, NULL, and Xmas scans exploit this asymmetry. They send packets that have no valid role in a normal TCP connection to a closed port, they should get an RST back. To an open port, they should get nothing — the packet is discarded.

The FIN scan sends a TCP packet with only the FIN flag set. Normally, FIN is sent to close an established connection. Sending FIN to a port with no established connection is technically illegal under TCP semantics.

The NULL scan sends a TCP packet with no flags set at all — every flag bit is zero. This is the most illegal of all, since no valid TCP packet has all flags empty.

The Xmas scan sends a TCP packet with FIN, PSH (Push), and URG (Urgent) all set simultaneously. The name comes from the image of a packet "lit up like a Christmas tree" with flags.

The critical limitation: Microsoft Windows does not implement RFC 793's behavior for these anomalous packets. Windows sends RST in response to FIN, NULL, and Xmas probes regardless of whether the port is open or closed. This means these three scan types are fundamentally non-functional against Windows targets — every port appears closed. Against Linux, Unix, and BSD targets, they work as described.

The practical value of these scans today is primarily in two scenarios. First, they can bypass very simple, stateless packet filters that only look for SYN packets as the indicator of new connections — a filter that blocks SYN packets might pass a FIN or NULL packet right through. Second, in certain compliance or forensic contexts, they provide evidence about a firewall's stateless versus stateful nature.

The ACK Scan (-sA) — Not for Finding Open Ports

The ACK scan is fundamentally different from everything discussed so far. It does not tell you whether ports are open or closed. Instead, it tells you which ports are filtered versus unfiltered — mapping the firewall's ruleset.

When you send a TCP packet with only the ACK flag set to a target, you are sending a packet that looks like it belongs to an already-established connection. The target knows nothing about an existing connection, so it would normally respond with RST to both open and closed ports — because an unexpected ACK from an unknown connection should be rejected.

A firewall is the variable. If a stateful firewall sees an ACK packet for a connection it has no record of, it drops the packet. If no firewall (or a stateless firewall) is in the path, the ACK reaches the host and gets a RST back.

So: RST response means unfiltered. No response means filtered.

By comparing SYN scan results (which ports are open) with ACK scan results (which ports are filtered), you can map the firewall's behavior precisely. A port that is open in the SYN scan but filtered in the ACK scan suggests a stateful firewall that tracks connections. A port that is filtered in both suggests a firewall blocking everything on that port. A port that is open in the SYN scan and unfiltered in the ACK scan suggests no firewall filtering on that port.

This intelligence is used for firewall evasion strategy — specifically, it reveals whether a stateless or stateful firewall is in use and which ports are controlled by firewall rules versus host-based filtering.

The Host Discovery Scan (-sn) — Finding Who Is Home

Before scanning ports on every IP in a /16 network (65,534 addresses), you need to know which of those addresses actually have live devices. The host discovery scan — run with the -sn flag — does exactly this: it determines which IP addresses have responding hosts without performing any port scanning.

On a local network (where the scanner and targets are on the same subnet), Nmap uses ARP (Address Resolution Protocol) requests when run as root. ARP operates at Layer 2 of the network model — below IP, below TCP, at the Ethernet level. An ARP request asks: "Which device on this local network has this IP address? Please tell me your MAC address."

ARP requests bypass host-based firewalls entirely. A host running Windows Firewall with all rules set to block incoming connections still must respond to ARP requests — otherwise it cannot communicate on the network at all. If you are on the same subnet as your targets, ARP-based host discovery is essentially immune to firewall evasion attempts.

The additional bonus: ARP responses include the target's MAC address, and the first three bytes of a MAC address are the OUI (Organizationally Unique Identifier) — a vendor code assigned by the IEEE. This means an ARP scan result tells you not just that a host is live, but what type of device it likely is. A MAC starting with 00:50:56 indicates a VMware virtual machine. B8:27:EB indicates a Raspberry Pi. 3C:D9:2B indicates an HP device. This vendor identification is immediate intelligence about the nature of the discovered hosts.

On a remote network (targets are across a router), ARP does not work — it is a local network protocol. Instead, Nmap uses a combination of techniques: ICMP echo requests (traditional ping), TCP SYN to port 443, TCP ACK to port 80, and ICMP timestamp requests. The logic is that even if a host blocks ICMP ping, it might respond to TCP connections on common ports. Using multiple probe types maximizes the probability of detecting live hosts even through partial filtering.

In a professional engagement, host discovery is run first with speed prioritized — using parallel probes and fast timeouts. The output (a list of live IPs) is saved to a file, which then becomes the input for full port scanning.

The Idle Scan (-sI) — True Anonymity in Port Scanning

The Idle scan is one of the most sophisticated techniques in network security research. It allows a penetration tester to scan a target without the target ever seeing the scanner's IP address. Instead, all probes appear to originate from a third, unrelated host — called the "zombie."

The technique relies on a subtle property of IP packets: each IP packet contains an ID field (IPID) — a number that is incremented for each packet a host sends. On some systems, particularly older ones, this counter increments predictably and globally — every packet the system sends, regardless of destination, increments the counter by one.

The attack works as follows. First, the scanner sends a probe to the zombie to learn its current IPID value. Then, the scanner sends a SYN packet to the target but spoofs the source address — making it look like the packet came from the zombie. If the target port is open, it sends a SYN-ACK back to the zombie (thinking the zombie initiated the connection). The zombie, receiving an unexpected SYN-ACK, sends a RST back to the target — and increments its IPID counter. When the scanner checks the zombie's IPID again, it has gone up by two (once for the scanner's initial probe, once for the zombie's RST to the target). If the target port is closed, it sends RST to the zombie, the zombie ignores it, and the IPID only increases by one.

The scanner never sends a single packet to the target from its own IP address. From the target's logs, only the zombie's IP appears. This technique is described in detail in Phrack Magazine and was a fundamental advance in network security research — it demonstrated that port scanning could be completely anonymous given the right conditions.

Modern operating systems no longer use globally sequential IPIDs (they use per-connection sequences or random values), which makes finding suitable zombie hosts increasingly difficult. The technique remains important to understand because it illustrates a deep principle: protocol properties that appear benign in isolation can be exploited in combination to achieve something their designers never intended.

Timing Options (-T0 through -T5) — Speed, Stealth, and the Art of the Tradeoff

Every penetration test involves a fundamental tension between speed and stealth. Scanning quickly means generating traffic rapidly, which means security devices can detect a clear pattern. Scanning slowly means staying below detection thresholds but taking potentially days or weeks to complete.

Nmap's timing templates are a high-level abstraction over a complex set of parameters that control how quickly probes are sent, how long to wait for responses, and how many probes to send in parallel.

Paranoid (-T0) sends one probe every five minutes. To scan a thousand ports at this speed takes over three days. No time-based correlation system — no SIEM, no IDS — can detect this as a scan because the packets are so spread out in time that they appear to be normal, sporadic traffic. This is used in extraordinarily sensitive engagements where detection must be avoided at all costs, or in research contexts where patience is available.

Sneaky (-T1) sends one probe every fifteen seconds. Still very slow, still effective at evading most time-based detection. A thousand-port scan takes around four hours.

Polite (-T2) slows down enough to avoid saturating the target network with probe traffic. It is more about being considerate of network bandwidth than about evasion. Useful when scanning production environments where causing network slowdowns would be noticed as operational disruption rather than security events.

Normal (-T3) is Nmap's default. It dynamically adjusts timing based on observed network conditions — if responses are coming back quickly, it sends probes faster; if there are timeouts, it slows down. A typical LAN scan completes in tens of seconds to a few minutes.

Aggressive (-T4) assumes a fast, reliable network. It shortens timeouts and increases parallelism significantly. This is the timing most commonly used in internal network assessments on LAN environments, CTF competitions, and lab environments where speed matters and stealth is not a concern.

Insane (-T5) pushes everything to the limit. Timeouts are extremely short, parallelism is maximized. The risk is that on slower or congested networks, probes time out before responses arrive, leading to ports being classified as filtered when they are actually open. Results from T5 scans should be verified with a slower timing setting if unusual results appear.

In a real engagement, the timing selection communicates something about the tester's strategy. External tests against a production web application might use T1 or T2 to stay under intrusion detection thresholds. Internal tests on a fast corporate LAN typically use T3 or T4. Red team engagements simulating advanced persistent threats use T0 or T1 during initial reconnaissance phases.

OS Detection, Version Detection, and the NSE — Completing the Picture

Beyond scan types, three additional Nmap capabilities deserve deep understanding because they transform raw port data into actionable intelligence.

Service and Version Detection (-sV) sends protocol-specific probes to each open port and compares the responses against a database of known service signatures. Instead of just knowing that port 8080 is open, version detection tells you it is running Apache Tomcat 9.0.37. That specific version number is then searchable against vulnerability databases — and Tomcat 9.0.37 happens to be vulnerable to CVE-2021-41079, a denial-of-service vulnerability. The jump from "port 8080 is open" to "this specific Tomcat version has a known CVE" happens because of version detection.

OS Detection (-O) works by analyzing subtle differences in how different operating systems implement TCP/IP. The initial sequence number (ISN) values that systems generate, the TCP options they include in packets, their response to unusual flag combinations, their IP TTL values, and their behavior with fragmented packets all vary between Windows, Linux, macOS, Cisco IOS, and other systems. Nmap maintains a database of these behavioral fingerprints. When it probes a host, it compares the observed behavior to the database and identifies the most likely OS — often with remarkable specificity, identifying not just "Linux" but "Linux 4.15-5.6" or not just "Windows" but "Windows 10 1903-1909."

The Nmap Scripting Engine (NSE) extends Nmap from a port scanner into a security assessment platform. Scripts are Lua programs that Nmap executes against discovered services. The default script set (-sC) runs safe, commonly useful scripts automatically. More targeted scripts can check for specific vulnerabilities: the smb-vuln-ms17-010 script checks whether a target is vulnerable to EternalBlue (the exploit used by WannaCry and NotPetya). The ssl-heartbleed script checks for the Heartbleed vulnerability. The snmp-brute script attempts to determine the SNMP community string using a wordlist.

3.2.3 Practice — Nmap in a Real Engagement Context

How a Real Scan Sequence Looks

In a professional engagement against a network like Pixel Paradise's, the scanning methodology follows a logical sequence that builds intelligence progressively.

You start with host discovery across the entire in-scope IP range. This tells you which addresses are live and what their MAC vendor identifiers are — giving immediate clues about device types. A block of addresses all resolving to VMware MAC prefixes suggests a virtualization environment. Cisco MAC addresses suggest network equipment. Raspberry Pi or unknown vendors might indicate IoT devices.

With the live host list in hand, you run an initial fast scan of the most common ports against all live hosts simultaneously. The goal is to get a broad overview quickly — which hosts are running web services, which are running Windows file sharing, which have SSH. This is typically a SYN scan of the top 1000 ports with version detection enabled.

For hosts that look particularly interesting based on the initial scan, you run a full port scan of all 65,535 TCP ports. This takes longer but ensures you do not miss services running on non-standard ports — a web application on port 8080, a database on port 5984, an administrative interface on port 9090.

Parallel to the TCP full scan, you run a targeted UDP scan of the most important UDP ports — 53, 67, 69, 123, 161, 162, 500, 514, and a few others depending on the environment. In a corporate network, SNMP is almost always present; whether it is secured is the question.

Finally, for each discovered service, you run targeted Nmap scripts and manual enumeration tools to extract detailed intelligence. A host with SMB open gets enum4linux run against it. A host with SNMP open gets snmpwalk. A host with a web server gets nikto and gobuster.

Nmap Commands in Professional Context

When running these scans, the commands follow patterns that you will use repeatedly throughout your career.

For initial host discovery across a network range, the command sends ARP requests on the local network and ICMP/TCP probes to remote networks, storing results to a file that subsequent scans use as input. The key options ensure no reverse DNS lookups slow things down (the -n flag tells Nmap not to bother converting IP addresses to hostnames during discovery, since we just need to know which hosts are alive).

For the initial broad port scan with service detection, the options combine a SYN scan (-sS) with service version detection (-sV) and the default NSE scripts (-sC). The output goes to all three formats simultaneously (-oA) so you have both human-readable results and machine-parseable XML for later processing.

For the full TCP port scan, the critical option is -p- which tells Nmap to scan all 65535 ports rather than just the top 1000. This takes longer but is essential for comprehensive coverage.

For UDP, the approach is to limit the scope to high-value ports and combine with version detection so Nmap sends protocol-specific probes rather than generic UDP packets — this dramatically improves accuracy.

# 1. Host Discovery
sudo nmap -sn -n 10.10.10.0/24 -oG live_hosts.gnmap
grep "Up" live_hosts.gnmap | awk '{print $2}' > live_hosts.txt

# 2. Initial Broad Scan (top 1000 ports + version + default scripts)
sudo nmap -sS -sV -sC -n --open -iL live_hosts.txt -oA initial_scan

# 3. Full TCP Port Scan (all 65535 ports)
sudo nmap -sS -p- -n -T4 --open -iL live_hosts.txt -oA full_tcp_scan

# 4. Targeted UDP Scan (key UDP services)
sudo nmap -sU -p 53,67,69,123,161,162,500,514,1900 -sV -n -iL live_hosts.txt -oA udp_scan

# 5. OS Detection (against confirmed live hosts)
sudo nmap -O -n -iL live_hosts.txt -oA os_detection

# 6. Comprehensive single-host scan (for high-interest targets)
sudo nmap -sS -sV -sC -O -p- -T4 --open --reason -n 10.10.10.50 -oA host_50_full

Reading Nmap Output — What the Results Tell You

When Nmap reports its findings, each piece of information carries specific intelligence. An open port 22 running OpenSSH 7.4 on an internal server means that SSH is accessible, and OpenSSH 7.4 is a version released in 2016 — cross-referencing against the CVE database reveals multiple vulnerabilities including CVE-2018-15919 (username enumeration) and CVE-2016-6515 (denial of service). Neither might be immediately exploitable, but the version date tells you the server has not been patched in years — which implies other services on this host are similarly outdated.

An open port 3389 (RDP) visible on the network from an external perspective is almost always a critical finding. RDP has been the vector for numerous high-impact attacks including BlueKeep (CVE-2019-0708) and DejaBlue (CVE-2019-1182). Even if these specific vulnerabilities are patched, RDP brute force and credential stuffing attacks remain highly effective.

A host running both SMB (port 445) and an old Windows version detected by OS fingerprinting is a potential EternalBlue target — the vulnerability at the core of WannaCry and NotPetya. Nmap's smb-vuln-ms17-010 script confirms this in seconds.

3.2.4 Types of Enumeration — Extracting Intelligence from Open Ports

The Difference Between Scanning and Enumeration

Port scanning tells you a door is open. Enumeration tells you what is behind that door, who has the keys, and what the security system looks like. It is the transition from reconnaissance to pre-exploitation intelligence gathering.

When you find port 445 open (SMB), you know Windows file sharing is running. Enumeration tells you the list of shared folders, the list of user accounts on the system, the domain it belongs to, the password policy (how complex passwords must be, how many failed attempts trigger a lockout), and whether guest access is allowed. This is not abstract security information — it is the specific intelligence needed to decide which attack approach to take.

Banner Grabbing — The Simplest Form of Enumeration

The simplest enumeration technique is banner grabbing. When most network services accept a new connection, they announce themselves by sending a text string identifying the software and its version. This string is called a "banner."

Imagine walking into a hotel and the receptionist says "Welcome to the Grand Hotel, this is Janet speaking." You have just learned where you are and who you are dealing with. Banner grabbing is the network equivalent — you connect to a service and it tells you who it is.

FTP (File Transfer Protocol, port 21) is particularly generous with information: connecting to an FTP server typically produces a response like 220 vsftpd 3.0.3 — directly revealing the software (vsftpd) and version (3.0.3). SSH servers send their version in the initial handshake: SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5 reveals not just OpenSSH 8.2p1 but the Ubuntu package version, which allows identification of the exact Ubuntu distribution and patch level.

HTTP servers include version information in the Server header of their responses: Server: Apache/2.4.41 (Ubuntu) or Server: Microsoft-IIS/10.0. The X-Powered-By header often adds a second layer: X-Powered-By: PHP/7.4.3.

Security-conscious server administrators often configure their services to remove or falsify these banners. Apache can be configured to report Server: Apache without the version, or even Server: Unknown. Nginx can be configured to send no Server header at all. This is security through obscurity — it does not fix any vulnerability, but it removes one easy source of information for attackers.

Even with falsified banners, though, the specific behavior of a service — which features it supports, how it formats error messages, the timing of its responses — often reveals the real software and version. Nmap's version detection database is built around these behavioral signatures, not just banner strings.

# The simplest banner grab — using netcat
nc -v target 21      # FTP — listen for the banner it sends immediately
nc -v target 22      # SSH — listen for the SSH identification string
nc -v target 25      # SMTP — listen for the SMTP greeting

# For HTTP, you need to send a request before getting a response
echo -e "HEAD / HTTP/1.0\r\n\r\n" | nc target 80

# OpenSSL for HTTPS — shows both the TLS handshake details and the HTTP response
openssl s_client -connect target:443 -quiet

# curl is often the most practical for HTTP/HTTPS banner grabbing
# -I means "send HEAD request and show only headers"
# -k means "do not verify SSL certificate" (useful for self-signed certs)
curl -I http://target
curl -I -k https://target

SNMP Enumeration — The Network's Open Book

SNMP deserves extensive discussion because it represents one of the most impactful enumeration findings possible. A single SNMP-enabled device with the default "public" community string can hand you a complete map of the internal network.

SNMP is a protocol designed for network management. It works on a simple query-response model: a management station sends queries to devices asking for specific pieces of management information, and the devices respond with that information. The information is organized in a hierarchical database called the Management Information Base (MIB).

The security model in SNMP versions 1 and 2c (the versions still predominantly in use) is embarrassingly simple: a shared password called a "community string." The read-only community string (almost universally "public" by default) allows any device that knows it to query any information the MIB exposes. The read-write community string ("private" by default) allows not just querying but modifying device configuration.

What can SNMP expose on a network device? Everything. The system description (device model, firmware version), the hostname, the list of all network interfaces and their IP addresses, the routing table (revealing network architecture), the ARP table (revealing every IP-to-MAC mapping the device has seen — effectively a list of all hosts on connected networks), the list of open TCP and UDP connections, BGP and OSPF neighbor information, interface bandwidth utilization, and error statistics.

On a server, SNMP exposes the complete list of running processes (including their command-line arguments, which sometimes include passwords), the list of installed software with version numbers, network connection state (equivalent to running netstat), and system performance data.

Finding SNMP open with default community strings on a core router during a penetration test is a major finding. It means the router's entire configuration is readable — including ACLs, routing policies, and potentially credentials stored in the configuration. If the read-write community string is also the default "private," the router can be reconfigured entirely without ever exploiting a single software vulnerability.

# First, confirm SNMP is running and find the community string
# onesixtyone is a fast SNMP community string brute forcer
onesixtyone -c /usr/share/seclists/Discovery/SNMP/common-snmp-community-strings.txt 10.10.10.50

# snmpwalk traverses the entire MIB tree under a given OID
# .1 is the root of the entire MIB tree — this walks everything
snmpwalk -v2c -c public 10.10.10.50 .1

# Target specific, high-value MIB branches:

# System information — hostname, OS description, contact, location
snmpwalk -v2c -c public 10.10.10.50 system

# All network interfaces and their IP addresses
snmpwalk -v2c -c public 10.10.10.50 interfaces
snmpwalk -v2c -c public 10.10.10.50 ipAddrTable

# ARP table — every IP-to-MAC the device has seen
snmpwalk -v2c -c public 10.10.10.50 ipNetToMediaTable

# Routing table — network topology
snmpwalk -v2c -c public 10.10.10.50 ipRouteTable

# Running processes (on servers)
snmpwalk -v2c -c public 10.10.10.50 hrSWRunName

# Installed software (on servers)
snmpwalk -v2c -c public 10.10.10.50 hrSWInstalledName

# snmp-check provides a much more readable formatted output
snmp-check -t 10.10.10.50 -c public -v 2c

SMB Enumeration — Windows Network Intelligence

SMB (Server Message Block) is the protocol Microsoft Windows uses for file sharing, printer sharing, and various inter-process communication functions. It has been central to Windows networking since the 1980s and remains the backbone of virtually every Windows enterprise environment today.

From a penetration testing perspective, SMB is one of the richest sources of intelligence on a Windows network — and also one of the most historically vulnerable protocols. EternalBlue, the exploit used by WannaCry ransomware in 2017, targeted SMB. Before that, MS08-067 (the "Conficker" vulnerability from 2008) also exploited SMB. Understanding SMB enumeration is fundamental to Windows network penetration testing.

What can SMB enumeration reveal? It starts with basic network information: the hostname of the target, the domain or workgroup it belongs to, and the operating system version. It then extends to the security configuration: whether null sessions are allowed (connecting without credentials), whether the Guest account is enabled, and the password policy (minimum length, complexity requirements, lockout threshold).

The password policy is particularly important. If the account lockout threshold is 5 (five failed attempts before lockout), you can attempt 4 passwords per account without triggering a lockout. If the lockout threshold is 0 (no lockout), you can attempt unlimited passwords. If the lockout observation window is 30 minutes (attempts reset every 30 minutes), you can attempt 4 passwords now, wait 30 minutes, attempt 4 more, and continue indefinitely. This intelligence directly determines the viability and strategy of password spray attacks.

SMB enumeration also reveals the list of shared folders. Shares with names ending in $ are "hidden" shares — they do not appear when browsing the network — but they are visible through enumeration. The default administrative shares C$, D$, ADMIN$, and IPC$ are present on every Windows machine. Access to ADMIN$ allows file upload to the Windows directory. Access to C$ allows reading and writing the entire C drive.

# enum4linux is the standard SMB enumeration tool for Linux
# -a performs a full enumeration (all options)
enum4linux -a 10.10.10.50

# What enum4linux tries to find:
# Workgroup/domain name
# NetBIOS hostname  
# OS version
# SMB shares (including hidden $ shares)
# User accounts
# Group memberships
# Password policy
# Printer information

# enum4linux-ng is the modern rewrite with better output formatting
enum4linux-ng -A 10.10.10.50

# smbclient lists shares — equivalent to "Network Places" in Windows
# The -L flag means "list shares"
# The // means "connect to this server"
# The -N flag means "no password" (null session)
smbclient -L //10.10.10.50 -N

# smbmap shows shares and your permission level on each
smbmap -H 10.10.10.50                          # Anonymous/null session
smbmap -H 10.10.10.50 -u john -p Password123  # Authenticated

# Nmap SMB scripts for specific intelligence gathering
nmap --script smb-enum-shares 10.10.10.50
nmap --script smb-enum-users 10.10.10.50
nmap --script smb-security-mode 10.10.10.50
nmap --script smb-vuln-ms17-010 10.10.10.50   # Check for EternalBlue
nmap --script "smb-vuln-*" 10.10.10.50        # All SMB vulnerability checks

DNS Enumeration (Active)

While passive DNS reconnaissance (covered in Section 3.1) uses third-party services to look up DNS records without touching the target's DNS servers, active DNS enumeration directly queries the target's DNS infrastructure.

The most impactful active DNS technique is the zone transfer — requesting that the target's DNS server send you its complete zone file. A DNS zone file contains every DNS record for a domain: every A record (hostname to IP mapping), every CNAME (alias), every MX (mail server), every TXT record, and every SRV record. A successful zone transfer is equivalent to getting a complete inventory of every named resource in the organization's DNS namespace.

Zone transfers are supposed to be restricted to authorized secondary DNS servers — the secondary servers that need to synchronize zone data from the primary. Misconfigured DNS servers allow zone transfer requests from any IP address, which means a penetration tester can retrieve the entire DNS database in seconds.

In practice, allowing unrestricted zone transfers is a recognized security misconfiguration that gets flagged in security assessments. Many organizations have corrected this. But it is always worth attempting, because the payoff of a successful zone transfer — an instant, complete inventory of the organization's DNS namespace — is significant.

Beyond zone transfers, active DNS enumeration includes brute force subdomain discovery (systematically querying the DNS server with a list of common subdomain names to find which ones resolve) and DNS walking for DNSSEC-enabled zones (which exposes the complete list of records in a zone through the NSEC record chain).

# Zone transfer attempt — always try this first
dig axfr targetco.com @ns1.targetco.com

# If zone transfer fails, brute force subdomains
# gobuster's DNS mode sends DNS queries for each word in the wordlist
gobuster dns -d targetco.com -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt -r 8.8.8.8

# dnsrecon is a comprehensive DNS enumeration tool
# -t std runs standard enumeration (SOA, NS, A, AAAA, MX, TXT, SRV)
# -t axfr attempts zone transfer
# -t brt brute forces subdomains
dnsrecon -d targetco.com -t std
dnsrecon -d targetco.com -t axfr
dnsrecon -d targetco.com -t brt -D /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt

# fierce does DNS reconnaissance specifically looking for network ranges
fierce --domain targetco.com

SMTP Enumeration — Verifying Email Accounts

SMTP (Simple Mail Transfer Protocol) is the protocol email servers use to send mail. Some SMTP servers support commands that allow querying whether a user account exists — functionality originally designed for legitimate purposes but now primarily used by spammers (to verify email lists) and penetration testers (to enumerate valid accounts for phishing or credential attacks).

The VRFY command asks the SMTP server to verify whether an email address or username is valid. A server responds with "250 OK" if the user exists or "550 No such user" if they do not.

The EXPN command expands a mailing list alias — asking the server to tell you all the email addresses that receive mail sent to a given alias. For example, EXPN all-staff@targetco.com might reveal dozens of individual employee email addresses.

The RCPT TO command is the most reliable verification method because it is part of normal mail delivery. An SMTP server must accept or reject each recipient address during mail delivery. Sending a test delivery attempt to RCPT TO: john.smith@targetco.com and observing whether the server accepts or rejects it reveals whether the account exists — even on servers that have disabled VRFY and EXPN.

Modern mail servers have largely disabled VRFY and EXPN for security reasons. But many older or poorly configured servers still respond. And the RCPT TO technique remains functional on many servers.

# Manual SMTP enumeration using netcat
# Connect to the SMTP server
nc -v target 25

# Once connected, you will see the SMTP banner (e.g., "220 mail.targetco.com ESMTP")
# Type these commands:
EHLO test.com                          # Introduce yourself, see supported commands
VRFY john.smith                        # Check if user exists
EXPN marketing                         # Expand a mailing list
QUIT                                   # Close connection

# smtp-user-enum automates this process across a user list
# -M VRFY specifies the method
# -U specifies the username file
# -t specifies the target
smtp-user-enum -M VRFY -U /usr/share/seclists/Usernames/Names/names.txt -t target
smtp-user-enum -M RCPT -U users.txt -D targetco.com -t target

# Nmap scripts for SMTP
nmap --script smtp-commands target     # List supported SMTP commands
nmap --script smtp-enum-users target   # Enumerate users
nmap --script smtp-open-relay target   # Check for open relay (allows anyone to send mail through this server)

HTTP and Web Service Enumeration

Web services are almost always present in modern penetration testing engagements — even networks that appear to be primarily Windows file sharing environments typically have some web-based management interface. Web service enumeration extends from basic banner grabbing into directory discovery, technology identification, and vulnerability scanning.

Directory and file brute forcing is the process of systematically requesting paths on a web server to discover hidden or unlisted content. A web application might have an administrative interface at /admin/, a backup file at /backup.zip, an API documentation page at /api/v2/docs, or a test file left by a developer at /test.php. None of these paths are linked from the main site — but they exist and are accessible. Directory brute forcing finds them.

The wordlists used for directory brute forcing are critical. SecLists — a curated collection of security-relevant wordlists maintained on GitHub — contains directory wordlists with hundreds of thousands of entries derived from real web applications and historical assessments. The most commonly used are the directory-list files in the /Discovery/Web-Content/ category.

Technology identification determines what CMS (Content Management System), framework, and programming language the web application uses. A WordPress site (identifiable by the /wp-content/, /wp-admin/, and /wp-login.php paths, as well as the ?p= parameter pattern in URLs) has a completely different vulnerability surface than a Laravel PHP application or a React+Node.js application. Tools like WhatWeb and the Wappalyzer browser extension analyze HTTP response headers, HTML structure, JavaScript includes, and cookie names to identify the technology stack with impressive accuracy.

Nikto is a web vulnerability scanner specifically designed for finding known issues in web server configurations. It checks for thousands of potentially dangerous files and programs, outdated server software, and configuration issues like directory listing being enabled, server headers exposing sensitive information, and missing security headers.

# Directory and file discovery — gobuster is the industry standard
# dir mode = directory/file brute forcing
# -u = target URL
# -w = wordlist
# -x = file extensions to check (appends these to each wordlist entry)
# -b = HTTP status codes to filter out (404 is the default not-found code)
gobuster dir -u http://10.10.10.50 \
             -w /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt \
             -x php,html,txt,bak,conf,xml \
             -b 404,403

# ffuf (Fuzz Faster U Fool) is faster and more flexible
# FUZZ is the placeholder for the wordlist value
ffuf -u http://10.10.10.50/FUZZ \
     -w /usr/share/seclists/Discovery/Web-Content/common.txt \
     -fc 404

# Technology identification
whatweb -v http://10.10.10.50      # -v for verbose output

# Nikto web vulnerability scanner
nikto -h http://10.10.10.50
nikto -h https://10.10.10.50 -ssl
nikto -h http://10.10.10.50 -o nikto_results.html -Format html

# Always manually check these files on every web server:
curl http://10.10.10.50/robots.txt          # Intentionally hidden paths
curl http://10.10.10.50/sitemap.xml         # Full site URL map
curl http://10.10.10.50/.well-known/security.txt  # Security contact info
curl -I http://10.10.10.50/                # Response headers reveal server info

3.2.5 Enumeration via Packet Crafting with Scapy

Why Scapy Matters Beyond Nmap

Nmap is a remarkable tool, but it is also a fixed tool — it sends predetermined probe sequences and interprets responses according to built-in logic. Scapy is the opposite: it is a Python library for constructing any network packet from scratch, at any layer, with any values in any field.

Think of Nmap as a Swiss Army knife — it has many tools, all expertly crafted for common tasks. Think of Scapy as a metalworking shop where you have raw steel and every tool imaginable — you can build any knife you want, including ones that have never existed before.

This matters for several reasons. Sometimes you need to test how a target responds to a specific, unusual packet — a packet with specific flag combinations, a specific sequence number, a malformed header. Nmap cannot send that. Scapy can. Sometimes you need to test a proprietary protocol that Nmap knows nothing about. Scapy can craft packets for any protocol you understand well enough to implement. Sometimes you need to automate a complex multi-packet interaction — send a packet, receive a response, make a decision based on the response, send a different packet based on that decision. Scapy can do this in Python.

For penetration testing enumeration specifically, Scapy is valuable for:

Implementing custom port scanning logic — for example, a SYN scan that sends probes at precisely calculated time intervals to evade timing-based IDS detection, adjusting the interval based on observed response patterns.

Building custom protocol interaction — if you discover a service running on an unusual port and need to speak its protocol, Scapy lets you construct and send the exact protocol-specific packets needed.

Testing firewall behavior — by crafting packets with specific, unusual combinations of flags, TTL values, or fragmentation patterns, you can probe exactly how a firewall processes different packet types.

Implementing attacks from academic research — security research papers frequently describe attacks in terms of specific packet sequences. Implementing these in Scapy is often the fastest path to a working proof-of-concept.

How Scapy Thinks About Packets

Scapy represents network packets as stacked objects, each representing one layer of the network model. The / operator stacks layers together, with lower layers on the left and higher layers on the right.

# This is how you think about a typical HTTPS request packet:
# Layer 2 (Ethernet) / Layer 3 (IP) / Layer 4 (TCP) / Application data
Ether() / IP() / TCP() / Raw("GET / HTTP/1.0\r\n\r\n")

# You can start at any layer — if you are routing (not on local LAN),
# you do not need Ethernet:
IP() / TCP()

# If you want to test ARP (purely local network):
Ether() / ARP()

# For ICMP ping:
IP() / ICMP()

Within each layer, every field has a default value, but you can override any field. The IP layer defaults to a TTL of 64, but you can change it. The TCP layer defaults to SYN flags, but you can set any combination of flags. You can set incorrect checksums to test how a target handles malformed packets. You can set impossible values to test bounds checking.

This level of control is what makes Scapy invaluable for advanced security testing and research — it treats the network protocol as what it actually is: a structured set of fields that can be set to any value.

Scapy in Enumeration Context

For the purposes of this module, Scapy is used in enumeration to supplement Nmap with custom probes. The most common use cases are:

Custom SYN scanning with fine-grained control. Nmap's SYN scan is excellent, but Scapy lets you control every aspect of the probe — the source port, the exact sequence number, the TCP window size, the timing between probes. This is useful when you need to craft probes that look like specific legitimate traffic to bypass application-layer inspection.

ARP scanning for local network host discovery. On a local LAN segment, sending ARP requests is the most reliable host discovery technique. Scapy makes it easy to build custom ARP scanning logic.

OS fingerprinting. Different operating systems respond differently to unusual packets. By crafting specific probe packets and analyzing the responses with Scapy, you can build a fingerprinting system tailored to your specific needs.

Protocol fuzzing. Sending packets with slightly malformed values to discover how a service handles invalid input — the foundation of fuzzing-based vulnerability discovery.

from scapy.all import *

# ARP Scan — Find all live hosts on the local network
# This sends an ARP broadcast to the entire subnet
# ARP cannot be blocked by host firewalls — it works at the Ethernet level
def arp_scan(network_range):
    # Ether(dst="ff:ff:ff:ff:ff:ff") = broadcast MAC (send to everyone)
    # ARP(pdst=network_range) = "Who has IP address X? Tell me your MAC"
    arp_request = Ether(dst="ff:ff:ff:ff:ff:ff") / ARP(pdst=network_range)

    # srp() = Send and Receive at the Ethernet (Packet) level
    # timeout=3 = wait 3 seconds for responses
    # verbose=False = suppress output (we handle display ourselves)
    answered, unanswered = srp(arp_request, timeout=3, verbose=False)

    live_hosts = []
    for sent_packet, received_packet in answered:
        ip = received_packet[ARP].psrc       # Source IP from ARP reply
        mac = received_packet[Ether].src     # Source MAC from Ethernet header
        live_hosts.append({'ip': ip, 'mac': mac})
        print(f"[+] {ip:<20} {mac}")

    return live_hosts

hosts = arp_scan("192.168.1.0/24")


# Custom SYN Scan — Port discovery with full control over probe construction
def syn_scan(target_ip, port_list):
    open_ports = []

    for port in port_list:
        # Construct the SYN packet
        # IP layer: set destination, let Scapy fill in source IP automatically
        # TCP layer: destination port, flags="S" means SYN only, random source port
        syn_packet = IP(dst=target_ip) / TCP(dport=port, flags="S", sport=RandShort())

        # sr1() = Send one packet, Receive one response
        # timeout=1 = wait 1 second for response before giving up
        response = sr1(syn_packet, timeout=1, verbose=False)

        if response is None:
            # No response at all = filtered
            continue

        if response.haslayer(TCP):
            tcp_flags = response[TCP].flags

            if tcp_flags == "SA":
                # SYN-ACK received = port is OPEN
                open_ports.append(port)
                print(f"[+] Port {port}: OPEN")

                # Send RST to cleanly close the half-open connection
                # Without this, the target keeps waiting for ACK completion
                rst = IP(dst=target_ip) / TCP(
                    dport=port,
                    sport=response[TCP].dport,
                    flags="R",
                    seq=response[TCP].ack
                )
                send(rst, verbose=False)

            elif "R" in str(tcp_flags):
                # RST received = port is CLOSED
                pass

    return open_ports

# Scan first 1024 ports
open_ports = syn_scan("10.10.10.50", range(1, 1025))


# ICMP Ping Sweep — More control than nmap -sn for specific scenarios
def icmp_sweep(network_range):
    # Create ICMP echo request packets for the entire range
    ping_packets = IP(dst=network_range) / ICMP()

    # sr() = Send multiple packets, Receive responses to all of them
    answered, unanswered = sr(ping_packets, timeout=2, verbose=False)

    print(f"\nHosts that responded to ICMP:")
    for sent, received in answered:
        print(f"[+] {received[IP].src} is alive (TTL={received[IP].ttl})")

        # TTL analysis for OS guessing
        ttl = received[IP].ttl
        if ttl <= 64:
            print(f"    → Likely Linux/Unix (TTL {ttl}, initial TTL probably 64)")
        elif ttl <= 128:
            print(f"    → Likely Windows (TTL {ttl}, initial TTL probably 128)")
        else:
            print(f"    → Likely network device (TTL {ttl}, initial TTL probably 255)")

icmp_sweep("192.168.1.1/24")

3.2.6 Lab Reference — Enumeration with Nmap

This section consolidates the professional enumeration workflow using Nmap — the commands and reasoning you would apply in a real lab or engagement environment against discovered targets.

The Enumeration Mindset

When you sit down at your attack machine with a list of open ports from your scan results, your mindset should be systematic: for each open port, identify the service, understand what information that service can expose, and use the right tools to extract that information.

The Pixel Paradise network (from the engagement scenario) contains diverse device types. A camera with an open HTTP port might have a default-credential web interface. A VoIP phone with SIP on port 5060 might expose extension numbers and authentication data. A Windows workstation with SMB open might have shares accessible without credentials. A network printer with SNMP might reveal the entire printer configuration including previously printed documents.

Treating enumeration as a generic "run these commands" exercise misses the point. Each discovered service is a potential source of intelligence that feeds into the attack phase. The question for every open port is: "What does this service know, and how can I extract it?"

Systematic Enumeration Commands

# ============================================================
# COMPLETE PROFESSIONAL ENUMERATION WORKFLOW
# ============================================================

# Target: 10.10.10.50 (discovered to have several open ports)

# STEP 1: Get full port and service detail for this specific host
sudo nmap -sS -sV -sC -O -p- -T4 --open --reason 10.10.10.50 -oA host_50_full

# This command combines:
# -sS    = SYN scan (fast, fewer logs)
# -sV    = Service and version detection
# -sC    = Default NSE scripts (runs category "default")
# -O     = OS detection
# -p-    = All 65535 TCP ports
# -T4    = Aggressive timing (good for internal LAN)
# --open = Only show open ports in output
# --reason = Show why Nmap classified each port as it did
# -oA    = Save all output formats (normal, XML, grepable)


# STEP 2: Based on what is open, run targeted enumeration

# If port 22 (SSH) is open:
nmap --script ssh-auth-methods --script-args="ssh.user=root" 10.10.10.50
nmap --script ssh2-enum-algos 10.10.10.50
# Look for: password authentication allowed (vs key-only), weak algorithm support

# If port 80/443 (HTTP/HTTPS) is open:
nikto -h http://10.10.10.50                     # Vulnerability scan
gobuster dir -u http://10.10.10.50 \
             -w /usr/share/seclists/Discovery/Web-Content/common.txt \
             -x php,html,txt
whatweb -v http://10.10.10.50                   # Technology fingerprint
curl -I http://10.10.10.50                      # Header analysis

# If port 21 (FTP) is open:
nmap --script ftp-anon 10.10.10.50              # Check anonymous login
nmap --script ftp-syst 10.10.10.50              # System information
ftp 10.10.10.50                                 # Manual connection attempt
# Username: anonymous, Password: anything@anything.com

# If port 25 (SMTP) is open:
nmap --script smtp-commands 10.10.10.50
nmap --script smtp-enum-users 10.10.10.50
smtp-user-enum -M VRFY -U /usr/share/seclists/Usernames/Names/names.txt -t 10.10.10.50

# If port 53 (DNS) is open:
dig axfr targetco.com @10.10.10.50              # Zone transfer attempt
dnsrecon -d targetco.com -t axfr -n 10.10.10.50

# If port 139/445 (SMB) is open:
enum4linux-ng -A 10.10.10.50                    # Full SMB enumeration
smbclient -L //10.10.10.50 -N
smbmap -H 10.10.10.50
nmap --script "smb-vuln-*" 10.10.10.50         # Vulnerability checks

# If port 161 (SNMP/UDP) is open:
onesixtyone -c /usr/share/seclists/Discovery/SNMP/common-snmp-community-strings.txt 10.10.10.50
snmp-check -t 10.10.10.50 -c public -v 2c      # If "public" works

# If port 389 (LDAP) is open:
ldapsearch -x -H ldap://10.10.10.50 -b "" -s base namingContexts  # Get base DN
ldapsearch -x -H ldap://10.10.10.50 -b "DC=targetco,DC=com" "(objectClass=*)"

# If port 3306 (MySQL) is open:
nmap --script mysql-info 10.10.10.50
nmap --script mysql-empty-password 10.10.10.50  # Check for no-password root
mysql -h 10.10.10.50 -u root --connect-timeout=5  # Manual login attempt

# If port 3389 (RDP) is open:
nmap --script rdp-enum-encryption 10.10.10.50
nmap --script rdp-vuln-ms12-020 10.10.10.50    # DoS vulnerability check

# If port 6379 (Redis) is open:
redis-cli -h 10.10.10.50 ping                  # Basic connectivity — no auth = critical finding
redis-cli -h 10.10.10.50 info server           # Server information
redis-cli -h 10.10.10.50 config get *          # All configuration

# If port 27017 (MongoDB) is open:
nmap --script mongodb-info 10.10.10.50
mongo --host 10.10.10.50 --eval "db.adminCommand({listDatabases:1})"  # No-auth check

3.2.7 Packet Inspection and Eavesdropping

What Eavesdropping Actually Means in a Network Context

When you are physically in a room with someone, eavesdropping means positioning yourself close enough to hear their conversation. Network eavesdropping is the same idea — positioning yourself on the network so that traffic flows through or past your machine, and capturing that traffic to analyze it.

The concept of "listening" to a network is both simpler and more complex than it sounds. Simpler because the data is already there — bytes are literally flowing through the network infrastructure, and if your network interface can see them, you can capture them. More complex because modern switched networks specifically prevent most ports from seeing traffic that is not addressed to them.

Understanding this requires understanding the difference between hubs and switches — two types of devices that connect computers in a local network.

A hub is a simple, old-fashioned device. When it receives a packet on one port, it forwards that packet out of every other port — broadcasting everything to everyone. Every computer connected to a hub can see every other computer's traffic, just by putting its network interface into "promiscuous mode" (accepting all packets, not just those addressed to it). Hubs are now obsolete, but they illustrate an important point: in an environment where all traffic is broadcast, eavesdropping is trivially easy.

A switch is intelligent. It learns the MAC address of the device connected to each port and builds a table (called the CAM table or MAC address table) that maps MAC addresses to port numbers. When it receives a packet destined for a specific MAC address, it forwards that packet only to the port where that MAC address is connected — not to every port. This isolation means a computer on port 5 normally cannot see traffic between computers on ports 3 and 7.

"Normally" is doing a lot of work in that sentence. Several techniques break this isolation:

Promiscuous mode still captures broadcast traffic (packets addressed to the broadcast MAC ff:ff:ff:ff:ff:ff, which go to all ports) and any traffic the switch mistakenly sends to your port. This is enough to capture ARP traffic and some discovery protocols but not individual unicast sessions between other hosts.

ARP Poisoning is the primary technique for eavesdropping on switched networks during authorized penetration tests. By poisoning the ARP caches of two communicating hosts, an attacker inserts themselves as the man-in-the-middle — all traffic between the two hosts flows through the attacker's machine, where it can be captured and analyzed before being forwarded to the real destination.

VLAN Hopping exploits misconfigurations in switch trunk port settings to send traffic onto VLANs (Virtual Local Area Networks) other than the one you are assigned to. VLANs are network segmentation technology — they create virtual separation between groups of devices on the same physical switch. A successful VLAN hop breaks this separation.

Switch port mirroring (SPAN port) is the legitimate, authorized version of eavesdropping. Many managed switches allow an administrator to configure a "SPAN port" that mirrors all traffic from specified ports to a monitoring port. During authorized penetration tests, the client may configure a SPAN port connected to your testing machine, giving you visibility into all traffic on the monitored segments without needing any attack technique.

What Is Capturable vs. What Is Encrypted

This is a critical distinction that determines the value of eavesdropping in any given environment.

For unencrypted protocols, packet capture is devastating. Telnet passes every keystroke in plaintext — capturing a Telnet session means seeing every command the user types and every response the server sends, including passwords entered at login prompts. FTP passes credentials in plaintext. HTTP passes all web traffic in plaintext — including POST bodies containing login form data, including session cookies that can be replayed to impersonate authenticated users.

SNMP v1 and v2c pass community strings in plaintext. Capturing a single SNMP query reveals the community string, which can then be used for extensive enumeration. POP3 and IMAP without TLS pass email credentials and content in plaintext.

For encrypted protocols, the content is protected, but metadata is still visible. You can see that a connection was made from IP address A to IP address B on port 443, at what time, for how long, and how many bytes were transferred — even if you cannot see the actual content. This metadata alone can be revealing: connections from an internal server to an unusual external IP address might indicate malware C2 communication, even though the content is encrypted.

The important nuance: encryption at the transport layer (TLS/SSL) protects content, but vulnerabilities in TLS itself (covered extensively in Section 3.1.13 on cryptographic flaws) can sometimes expose encrypted traffic to decryption. SSL stripping attacks can downgrade HTTPS connections to HTTP in some scenarios, removing the encryption protection entirely.

ARP Poisoning — The MitM Foundation

ARP (Address Resolution Protocol) is the protocol that translates between IP addresses and MAC addresses on a local network. When computer A wants to send a packet to computer B (IP 192.168.1.2), it first broadcasts an ARP question: "Who has 192.168.1.2? Tell 192.168.1.1." Computer B responds: "192.168.1.2 is at MAC AA:BB:CC:DD:EE:FF." Computer A caches this mapping in its ARP table and uses it for all future packets to that IP.

The critical weakness: ARP has no authentication whatsoever. Any device can send an ARP reply claiming any IP-to-MAC mapping, and the receiving device will update its ARP cache with the new information — even without having asked a question. These are called "gratuitous ARP replies."

ARP poisoning exploits this by sending fake gratuitous ARP replies to both sides of a communication:

To the victim (say, a workstation at 192.168.1.100): "192.168.1.1 (the gateway) is at MAC AA:BB:CC:11:22:33 (the attacker's MAC)."

To the gateway (192.168.1.1): "192.168.1.100 (the workstation) is at MAC AA:BB:CC:11:22:33 (the attacker's MAC)."

Now when the workstation tries to send traffic to the internet, it sends it to the attacker's MAC instead of the gateway's MAC. The attacker receives the traffic, captures it, and then forwards it to the real gateway — so the communication still works from the victim's perspective. The victim sees no disruption. The attacker sees everything.

This must only be performed with explicit authorization on networks you have permission to test. ARP poisoning can cause network disruption if IP forwarding is not properly enabled on the attacking machine — traffic gets captured but not forwarded, causing apparent network outages for the victim.

# ARP poisoning using arpspoof (from the dsniff package)
# You need TWO terminal windows running simultaneously

# Terminal 1: Tell the VICTIM that YOU are the GATEWAY
# -i eth0 = use this network interface
# -t 192.168.1.100 = poison this target's ARP cache
# 192.168.1.1 = claim to be this IP (the gateway)
sudo arpspoof -i eth0 -t 192.168.1.100 192.168.1.1

# Terminal 2: Tell the GATEWAY that YOU are the VICTIM
sudo arpspoof -i eth0 -t 192.168.1.1 192.168.1.100

# CRITICAL: Enable IP forwarding so traffic actually passes through
# Without this, all victim traffic is captured but not forwarded
# — the victim's internet connection appears to stop working
echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward

# Bettercap — the modern, more powerful alternative
sudo bettercap -iface eth0

# Inside bettercap's interactive console:
# Discover hosts on the network
net.probe on
# Wait a moment, then show discovered hosts
net.show
# Set target for ARP poisoning
set arp.spoof.targets 192.168.1.100
# Start ARP poisoning
arp.spoof on
# Start capturing traffic
net.sniff on

3.2.8 Practice — Packet Inspection in a Real Scenario

A Real-World Eavesdropping Scenario

To understand why packet inspection and eavesdropping matters in a penetration test, consider this realistic scenario: you are conducting an internal network assessment for a financial company. Your scan has found a /24 network segment with mixed devices. You have established a man-in-the-middle position on this segment by ARP poisoning the default gateway.

With traffic flowing through your machine, you start Wireshark and look at what you can see. Within minutes, a striking picture emerges. Periodic packets are flowing from one server to a network printer — and they are SNMP queries, completely unencrypted, revealing the community string "private" in plaintext. That "private" community string is the read-write community string for the printer's management interface. You now have full management access to every printer on the network — including the ability to retrieve previously printed documents from the printer's internal storage.

Five minutes later, you see FTP traffic between an employee's workstation and an internal file server. FTP sends credentials in plaintext. You can read: "USER jsmith" and "PASS Summer2024!" — a credential that, because of password reuse, might also work for the company's VPN, email system, or Active Directory.

Later in the capture, a developer connects to their development database and the connection string — including server address, database name, username, and password — passes in cleartext in the HTTP traffic from a web application still in HTTP (not HTTPS) during development.

None of this required exploiting a single software vulnerability. It was all available in the network traffic, captured entirely legally within the scope of the authorized penetration test. This scenario illustrates precisely why encryption in transit matters so much — and why its absence is always a critical finding in a penetration test report.

Wireshark Display Filters in Practice

When analyzing captured traffic, the challenge is not capturing packets — it is finding the relevant packets among potentially millions. A busy network generates enormous amounts of traffic, and Wireshark display filters are the primary tool for cutting through the noise.

Think of display filters as SQL WHERE clauses for network traffic. You can filter on any field of any protocol with equality, inequality, range, and string match operators. You can combine conditions with AND, OR, and NOT. The filter language is intuitive once you understand the pattern: protocol.field_name == value.

The most practically useful filters for penetration testing fall into a few categories:

For finding authentication traffic, you are looking for HTTP POST requests (which carry form data including login credentials), FTP credential commands (USER and PASS), and cleartext authentication in older protocols.

For finding data of interest, you are searching packet contents for keywords like "password," "credential," "admin," or specific data types relevant to the engagement (credit card numbers, social security numbers, PHI).

For credential capture via NTLM, SMB traffic on a Windows network carries NTLM authentication hashes during the login process. These hashes are not cleartext passwords, but they can be cracked offline or used directly in pass-the-hash attacks.

# Launch Wireshark from command line
sudo wireshark &

# Or use the command-line version for remote/headless scenarios
sudo tshark -i eth0

# Capture to file for later analysis
sudo tshark -i eth0 -w capture.pcap

# Read from capture file
tshark -r capture.pcap

# Apply display filter while reading
tshark -r capture.pcap -Y "http.request.method == POST"

In Wireshark's display filter bar, these filters are typed directly:

# Show only HTTP POST requests (login form submissions)
http.request.method == "POST"

# Show all HTTP traffic containing the word "password" anywhere
http contains "password"

# Show traffic between two specific hosts
ip.addr == 192.168.1.100 && ip.addr == 192.168.1.200

# Show FTP authentication commands
ftp.request.command == "USER" || ftp.request.command == "PASS"

# Show all DNS queries (not responses) — what are users looking up?
dns.flags.response == 0

# Show NTLM authentication exchanges in SMB traffic
ntlmssp

# Show all traffic to or from a specific subnet
ip.addr == 10.10.10.0/24

# Show everything EXCEPT your SSH connection (so you do not pollute the capture)
not (tcp.port == 22 && ip.addr == your_ip)

# Show only TCP connection initiations (SYN packets without ACK)
tcp.flags.syn == 1 && tcp.flags.ack == 0

# Find large data transfers that might be exfiltration
tcp.len > 10000

# Show all SNMP traffic (where community strings live)
snmp

# Show all Telnet traffic (fully cleartext)
telnet

The "Follow TCP Stream" feature in Wireshark is particularly powerful — right-clicking any packet in a session and selecting "Follow → TCP Stream" reassembles the entire conversation between the two parties and displays it as human-readable text, exactly as it passed over the wire. A login session's credentials, a file transfer's contents, a database query and its results — all visible in a single coherent view.

3.2.9 Packet Crafting with Scapy — Full Professional Reference

The Philosophy of Packet Crafting

To use Scapy well, you need to understand what you are actually doing at a technical level. Network communication is, at its foundation, a set of structured byte sequences. An IP packet is not magic — it is a sequence of bytes with specific fields at specific positions. The first byte contains version and header length information. Bytes 2-3 are the total length. Bytes 8-9 are the TTL and protocol. Bytes 12-15 are the source IP address. Bytes 16-19 are the destination IP address.

Every protocol is defined by a specification (usually an RFC — Request for Comments document) that precisely defines which fields appear at which byte positions, how large each field is, and what values are valid. TCP's SYN flag lives in bit 1 (zero-indexed) of byte 13 in the TCP header. ACK is in bit 4. FIN is in bit 0. Scapy knows all of this — it encodes every RFC-defined protocol as a Python class where each field is a named attribute.

When you set TCP(flags="S") in Scapy, you are setting the SYN bit in that byte of the TCP header. When you set IP(ttl=128), you are writing 128 into byte 8 of the IP header. Scapy translates your Python attribute assignments into the correct byte positions in the packet before sending it.

This is why Scapy is so powerful: it is a protocol-aware byte constructor that understands every standard protocol but does not constrain you to valid values. You can set TTL to 0, flags to illegal combinations, sequence numbers to any value — because sometimes security testing requires sending exactly those invalid packets to see how a target responds.

Scapy Architecture — Send and Receive Functions

Scapy has four primary functions for sending packets and receiving responses, and understanding which to use when is important:

send() sends a packet at Layer 3 (IP level) and does not wait for a response. Use this when you want to send a packet and do not care about the response — like sending a RST to close a half-open connection.

sendp() sends a packet at Layer 2 (Ethernet level) and does not wait for a response. Use this when you need to send a raw Ethernet frame, such as in ARP operations.

sr() sends packets at Layer 3 and waits for responses. It returns two lists: answered (matched request-response pairs) and unanswered (requests with no response). Use this for scanning multiple targets simultaneously and processing all responses.

sr1() sends one packet at Layer 3 and waits for one response. Returns the single response packet (or None if no response). Use this when you are testing a single port or target and want to check the response before proceeding.

srp() and srp1() are the Layer 2 equivalents of sr() and sr1(). Use these for ARP operations and any other scenarios requiring Ethernet-level packet construction.

from scapy.all import *

# ============================================================
# BUILDING AND INSPECTING PACKETS
# ============================================================

# See all fields available in a layer
ls(IP)           # Shows every IP header field and its default
ls(TCP)          # Shows every TCP header field and its default
ls(UDP)
ls(ICMP)
ls(ARP)
ls(DNS)

# Build a basic packet and inspect it
packet = IP(dst="10.10.10.50") / TCP(dport=80, flags="S")

# .show() displays the packet in a human-readable structured format
packet.show()

# .show2() is like .show() but computes checksums and lengths first
packet.show2()

# hexdump() shows the raw bytes
hexdump(packet)

# len() shows total packet size in bytes
print(f"Packet size: {len(packet)} bytes")


# ============================================================
# ICMP OPERATIONS
# ============================================================

# Simple ping
target = "10.10.10.50"
ping_pkt = IP(dst=target) / ICMP()

# sr1() sends the packet and waits for ONE response
response = sr1(ping_pkt, timeout=2, verbose=False)

if response:
    print(f"Host {target} is alive")
    print(f"TTL: {response[IP].ttl}")
    print(f"Response type: {response[ICMP].type}")  # 0 = echo reply
else:
    print(f"No response from {target}")

# Ping sweep of entire subnet
def ping_sweep(network):
    # IP(dst=network) with a CIDR range creates a packet for each IP
    packets = IP(dst=network) / ICMP()
    answered, unanswered = sr(packets, timeout=2, verbose=False)

    print(f"\nResults for {network}:")
    print(f"Live hosts: {len(answered)}")
    print(f"No response: {len(unanswered)}")

    for sent_pkt, reply_pkt in answered:
        ttl = reply_pkt[IP].ttl
        src = reply_pkt[IP].src

        # OS estimation from TTL
        if ttl <= 64:
            os_guess = "Linux/Unix"
        elif ttl <= 128:
            os_guess = "Windows"
        else:
            os_guess = "Network device"

        print(f"  [+] {src:<20} TTL={ttl:<5} Probably: {os_guess}")

ping_sweep("10.10.10.0/24")


# ============================================================
# TCP OPERATIONS
# ============================================================

# Full SYN scan implementation
def custom_syn_scan(target, ports):
    print(f"\nScanning {target} ports: {min(ports)}-{max(ports)}")
    open_ports = []
    closed_ports = []
    filtered_ports = []

    for port in ports:
        # Build SYN packet
        # RandShort() generates a random source port (avoids confusion)
        pkt = IP(dst=target) / TCP(dport=port, flags="S", sport=RandShort())

        response = sr1(pkt, timeout=1, verbose=False)

        if response is None:
            filtered_ports.append(port)

        elif response.haslayer(TCP):
            flags = response[TCP].flags

            # "SA" = SYN-ACK = port is OPEN
            if flags == "SA" or flags == 0x12:
                open_ports.append(port)

                # Send RST to avoid leaving half-open connections on target
                # Half-open connections consume resources on the target
                rst = IP(dst=target) / TCP(
                    dport=port, 
                    sport=response[TCP].dport,
                    flags="R",
                    seq=response[TCP].ack
                )
                send(rst, verbose=False)

            # "RA" or "R" = RST = port is CLOSED
            elif "R" in str(flags):
                closed_ports.append(port)

    print(f"\nOpen ports: {open_ports}")
    print(f"Filtered ports: {len(filtered_ports)}")
    print(f"Closed ports: {len(closed_ports)}")
    return open_ports

common_ports = [21, 22, 23, 25, 53, 80, 110, 135, 139, 143, 443, 445, 3306, 3389, 5900, 8080]
open_ports = custom_syn_scan("10.10.10.50", common_ports)


# ============================================================
# UDP OPERATIONS
# ============================================================

# UDP scan with protocol-specific payloads
def udp_scan(target, port_payloads):
    """
    port_payloads = dict of {port: payload_bytes}
    Providing protocol-specific payloads dramatically improves accuracy
    """
    results = {}

    for port, payload in port_payloads.items():
        if payload:
            pkt = IP(dst=target) / UDP(dport=port) / Raw(payload)
        else:
            pkt = IP(dst=target) / UDP(dport=port)

        response = sr1(pkt, timeout=2, verbose=False)

        if response is None:
            results[port] = "open|filtered"
        elif response.haslayer(UDP):
            results[port] = "open"
        elif response.haslayer(ICMP):
            icmp_type = response[ICMP].type
            icmp_code = response[ICMP].code
            if icmp_type == 3 and icmp_code == 3:
                results[port] = "closed"  # ICMP Port Unreachable
            else:
                results[port] = "filtered"

    for port, state in results.items():
        if state == "open":
            print(f"[+] UDP {port}: {state}")
        elif state == "open|filtered":
            print(f"[?] UDP {port}: {state}")

    return results

# Protocol-specific UDP payloads
udp_targets = {
    53:  b"\x00\x00\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x07version\x04bind\x00\x00\x10\x00\x03",  # DNS version query
    161: b"\x30\x26\x02\x01\x01\x04\x06public\xa0\x19\x02\x04\x71\xb4\xb5\x60\x02\x01\x00\x02\x01\x00\x30\x0b\x30\x09\x06\x05\x2b\x06\x01\x02\x01\x05\x00",  # SNMPv2c GET
    123: b"\x1b" + b"\x00" * 47,  # NTP client request
    69:  b"\x00\x01test.txt\x00netascii\x00",  # TFTP read request
}

udp_scan("10.10.10.50", udp_targets)


# ============================================================
# ARP OPERATIONS
# ============================================================

# ARP scan of local network segment
def arp_scan(network):
    # Ether broadcast + ARP who-has
    request = Ether(dst="ff:ff:ff:ff:ff:ff") / ARP(pdst=network)

    # srp() for Layer 2 (need Ethernet header for ARP)
    answered, _ = srp(request, timeout=3, verbose=False)

    hosts = []
    print(f"\nARP scan results for {network}:")
    print(f"{'IP Address':<18} {'MAC Address':<20} {'Vendor Hint'}")
    print("-" * 60)

    for _, response in answered:
        ip = response[ARP].psrc
        mac = response[Ether].src

        # OUI lookup (first 3 octets of MAC)
        oui = mac[:8].upper()
        vendor = {
            "00:50:56": "VMware",
            "00:0C:29": "VMware Workstation",
            "08:00:27": "VirtualBox",
            "B8:27:EB": "Raspberry Pi",
            "00:1A:A0": "Dell",
            "00:1E:4F": "Dell",
            "DC:A6:32": "Raspberry Pi 4",
            "3C:D9:2B": "Hewlett Packard",
            "AC:DE:48": "Apple",
        }.get(oui, "Unknown")

        print(f"{ip:<18} {mac:<20} {vendor}")
        hosts.append({'ip': ip, 'mac': mac, 'vendor': vendor})

    return hosts

hosts = arp_scan("192.168.1.0/24")

3.2.10 Network Sniffing with Wireshark — The Complete Guide

Wireshark's Role in Penetration Testing

Wireshark is the world's most widely used network protocol analyzer. It captures network packets in real time — or reads them from previously saved capture files — and provides deep inspection of every layer of every protocol in those packets.

For a penetration tester, Wireshark serves several distinct purposes depending on the phase of the engagement. During active reconnaissance, it lets you analyze the traffic patterns on a network segment you have access to — understanding what protocols are in use, what the "normal" traffic looks like, and identifying interesting communications. During exploitation, it helps verify that your payloads are reaching the target correctly and that responses are coming back. During a man-in-the-middle attack, it captures the decrypted traffic flowing through your machine.

But perhaps most importantly, Wireshark is a learning tool. There is no better way to understand a protocol than to capture real traffic using that protocol and examine each packet in detail. Want to understand exactly how the TLS handshake works? Capture an HTTPS connection and walk through it. Want to understand what NTLM authentication looks like? Capture a Windows login and follow the SMB authentication sequence. The ability to see real protocols in action transforms abstract concepts into concrete, observable behavior.

Capturing Traffic with Wireshark

When you launch Wireshark with sufficient privileges (root or a user in the wireshark group on Linux), you are presented with a list of network interfaces. Selecting an interface and clicking the shark-fin "Start Capturing" button begins recording all packets that the interface receives.

The key question is which interface to use. On a typical Linux penetration testing machine:

eth0 or ens33 is the primary Ethernet interface — this is your wired network connection. Choose this for capturing traffic on a wired network segment.

wlan0 is the wireless interface in normal mode — captures wireless management traffic. For capturing wireless data traffic, you need to put the interface into monitor mode first (using airmon-ng start wlan0).

lo is the loopback interface — captures traffic between processes on your own machine. Useful for testing locally running services.

After capturing, you save the capture as a .pcap or .pcapng file for later analysis. This is important for documentation — your captured packets are evidence for your penetration test report.

The Wireshark Interface

The Wireshark interface has three main panels that work together.

The Packet List panel at the top shows each captured packet as one row, with columns for packet number, timestamp, source IP, destination IP, protocol, length, and a brief description. Colors indicate protocol types (green for TCP, blue for DNS, yellow for ARP, etc.) and anomalies (red for errors).

The Packet Details panel in the middle shows the dissected structure of the selected packet — every layer expanded into its component fields. This is where you read the actual values of IP TTL, TCP flags, HTTP headers, DNS query names, and every other protocol field.

The Packet Bytes panel at the bottom shows the raw bytes of the selected packet in hexadecimal on the left and ASCII on the right. When you click a field in the Packet Details panel, the corresponding bytes are highlighted in the Packet Bytes panel — directly showing you which bytes in the raw packet encode which protocol field.

Wireshark Display Filters — The Complete Practical Guide

Display filters are what make Wireshark usable on real captures. A busy corporate network generates thousands of packets per second — without filtering, finding anything in the noise is impossible. The filter language allows precise, specific queries.

The syntax follows a consistent pattern: protocol.field_name operator value. For example, ip.src is the source IP address field in the IP protocol layer. tcp.flags.syn is the SYN flag field in the TCP layer. http.request.method is the method field in HTTP requests.

Operators include == (equals), != (not equals), > and < (greater/less than), contains (string contains substring), and matches (regular expression match). Conditions combine with && (AND), || (OR), and ! (NOT).

# ============================================================
# ESSENTIAL DISPLAY FILTERS FOR PENETRATION TESTING
# ============================================================

# --- Credential Hunting ---

# HTTP POST requests — these carry login form data
http.request.method == "POST"

# HTTP traffic containing "password" anywhere in the packet
http contains "password"

# HTTP Authorization header (Basic Auth sends base64-encoded credentials)
http.authorization

# FTP login commands — these are always cleartext
ftp.request.command == "USER" || ftp.request.command == "PASS"

# All Telnet traffic (completely cleartext protocol)
telnet

# SNMP with community strings visible
snmp

# NTLM authentication exchanges (Windows credential material)
ntlmssp


# --- Traffic Analysis ---

# All traffic between two specific hosts
ip.addr == 192.168.1.100 && ip.addr == 10.10.10.50

# All traffic from one source to anywhere
ip.src == 192.168.1.100

# Traffic from an entire subnet
ip.src == 192.168.1.0/24

# Traffic on a specific port
tcp.port == 8080
udp.port == 161

# All DNS queries (who is this machine looking up?)
dns.flags.response == 0

# DNS queries for a specific domain
dns.qry.name contains "internal"
dns.qry.name == "secretproject.targetco.local"

# All ARP traffic (see who is doing network discovery)
arp

# ARP requests only (not replies)
arp.opcode == 1


# --- TCP Analysis ---

# New connection initiations (SYN without ACK = start of handshake)
tcp.flags.syn == 1 && tcp.flags.ack == 0

# Connection resets (can indicate scanning, blocking, or errors)
tcp.flags.reset == 1

# TCP retransmissions (network issues or dropped packets)
tcp.analysis.retransmission

# Zero window (receiver cannot accept more data — resource exhaustion)
tcp.analysis.zero_window

# Large packets (potential file transfer, could be exfiltration)
tcp.len > 10000 || udp.length > 1000


# --- Protocol-Specific ---

# SMB traffic (Windows file sharing, authentication)
smb || smb2

# SSH traffic (identifies servers offering remote access)
ssh

# RDP traffic (Remote Desktop)
rdp

# HTTP error responses (reveals application behavior under stress)
http.response.code >= 400

# All HTTPS (encrypted, but metadata like SNI and cert subject visible)
tls

# TLS handshake packets (reveals SNI — Server Name Indication — which
# hostname the client is trying to connect to, even in encrypted traffic)
tls.handshake.type == 1


# --- Investigation Shortcuts ---

# Everything except your own SSH connection to avoid noise
!(tcp.port == 22 && ip.addr == 192.168.1.50)

# Find packets referencing specific string (useful for credential hunting)
frame contains "admin"
frame contains "password"
frame contains "secret"

The Follow Stream Feature — Seeing Conversations

The most powerful single feature in Wireshark for credential capture and data analysis is "Follow Stream." When you right-click any packet in a session and choose "Follow → TCP Stream" (or "HTTP Stream" or "TLS Stream"), Wireshark reassembles the entire conversation between the two parties — every packet, in order, reconstructed into a coherent dialogue.

The display shows client-to-server traffic in one color and server-to-client traffic in another. For an HTTP login transaction, you see the complete HTTP request including all headers and the POST body (which contains the username and password), followed by the complete HTTP response.

For FTP, you see the entire session: the server's greeting, the client's USER command with the username, the server's "331 Password required" response, the client's PASS command with the password in cleartext, and all subsequent file transfer commands.

For Telnet, you see every character typed and every character the server sent back — including the login: and Password: prompts and the keystrokes used to answer them.

This view is often what goes directly into a penetration test report: a screenshot of "Follow TCP Stream" showing cleartext credentials is unambiguous evidence of a critical security finding.

tcpdump — When Wireshark Is Not Available

Wireshark requires a graphical interface. In many real-world penetration testing scenarios — remote servers accessed via SSH, headless Linux VMs, cloud instances — there is no GUI available. tcpdump is the command-line packet capture tool that provides the same capture capability in pure text form.

tcpdump captures packets and displays them as text, or saves them to .pcap files that can be analyzed in Wireshark later. This is the most common professional workflow: use tcpdump to capture on a remote machine over SSH, then transfer the .pcap file to your local machine and analyze it in Wireshark.

tcpdump's filtering syntax is BPF (Berkeley Packet Filter) — a powerful filter language also used by the Linux kernel's firewall subsystem. It is different from Wireshark's display filter syntax but follows similar logical principles.

# Capture all traffic on eth0 to a file
sudo tcpdump -i eth0 -w /tmp/capture.pcap

# Capture with a filter — only HTTP and HTTPS traffic
sudo tcpdump -i eth0 -w /tmp/web_traffic.pcap 'port 80 or port 443'

# Capture to file with timestamps and without DNS resolution
# -n = no DNS resolution (faster, avoids polluting capture with DNS queries)
# -nn = no DNS resolution AND no port name resolution (shows 80, not http)
sudo tcpdump -i eth0 -n -w /tmp/capture.pcap

# Watch traffic in real time — print packets in ASCII
# -A = print packet in ASCII (great for seeing cleartext credentials)
sudo tcpdump -i eth0 -A port 80

# Watch traffic in real time — print in hex and ASCII
# -X = hex + ASCII
sudo tcpdump -i eth0 -X port 80

# Capture only FTP authentication (always cleartext)
sudo tcpdump -i eth0 -A 'port 21'

# Capture SNMP traffic to read community strings
sudo tcpdump -i eth0 -A 'udp port 161'

# Capture between specific hosts
sudo tcpdump -i eth0 'host 192.168.1.100 and host 192.168.1.1'

# Capture everything EXCEPT SSH (so your own session does not appear)
sudo tcpdump -i eth0 'not port 22'

# Capture for a time limit then stop
sudo timeout 60 tcpdump -i eth0 -w /tmp/sixty_second_capture.pcap

# Read and analyze a pcap file with tcpdump
tcpdump -r /tmp/capture.pcap
tcpdump -r /tmp/capture.pcap -A 'port 80'

# Extract HTTP POST bodies from a capture file
tcpdump -r /tmp/capture.pcap -A 'tcp port 80' | grep -A5 "POST"

# Capture with rotation — new file every 100MB, keep last 5 files
# Prevents capture files from growing indefinitely
sudo tcpdump -i eth0 -C 100 -W 5 -w /tmp/capture.pcap

Putting It Together — A Complete Eavesdropping Workflow

The complete workflow for network eavesdropping in an authorized internal penetration test looks like this:

First, you verify you have network access to the target segment — either through physical access (your testing machine is connected to the network) or through a compromised machine that gives you routing access.

Second, if on a switched network without a SPAN port, you set up ARP poisoning to redirect traffic through your machine. You enable IP forwarding so the traffic continues to flow normally — the goal is to intercept, not disrupt.

Third, you start capturing with either Wireshark or tcpdump. If using tcpdump on a remote machine, you start it with output to a file.

Fourth, you let the capture run for a sufficient period to observe normal network activity. Business hours produce much richer captures than off-hours because users are actually doing things — browsing intranet sites, accessing file shares, authenticating to services.

Fifth, you analyze the capture using display filters in Wireshark — hunting for credentials, sensitive data, interesting protocol interactions, and evidence of security weaknesses.

Finally, you document your findings with packet captures and screenshots as evidence, clean up your ARP poisoning to restore normal network operation, and include the findings in your penetration test report.

Every cleartext credential found, every unencrypted sensitive data transmission, every protocol vulnerability revealed through traffic analysis is a documented finding with cryptographically robust evidence: the actual captured packets.

Module 3 — Section 3.3: Understanding the Art of Performing Vulnerability Scans

CompTIA PenTest+ / Ethical Hacking Certification Series
Professional Reference Guide — GitHub Edition
Written to build deep understanding, not just tool familiarity.

3.3.1 Overview — What Vulnerability Scanning Actually Is
3.3.2 How a Typical Automated Vulnerability Scanner Works
3.3.3 The CVE and CVSS Systems — The Language of Vulnerability
3.3.4 Types of Vulnerability Scans
3.3.5 The Major Vulnerability Scanning Tools — Deep Comparison
3.3.6 Vulnerability Scanning with Kali Tools — Practical Reference
3.3.7 Challenges to Consider When Running a Vulnerability Scan
3.3.8 Vulnerability Scanning in the Penetration Testing Lifecycle
3.3.9 Interpreting and Acting on Scan Results

3.3.1 Overview — What Vulnerability Scanning Actually Is

The Gap Between Knowing What Is There and Knowing What Is Wrong

After active reconnaissance, you know what is on the network: which hosts are live, which ports are open, what services and versions are running, and what operating systems those hosts use. This is enormous progress — but it is only inventory. Knowing that Apache Tomcat 9.0.37 is running on port 8080 does not automatically tell you whether that specific version has any known security weaknesses, and if so, how severe they are and whether they are exploitable from the network.

Vulnerability scanning bridges this gap. It takes the inventory produced by reconnaissance and cross-references it against databases of known security weaknesses, asking a very specific question for each discovered asset: "Does this system or service have any known vulnerabilities that an attacker could exploit?"

The answer comes from matching what the scanner observes — software versions, protocol behaviors, configuration responses, service banners — against a continuously maintained database of known vulnerabilities. This database, at its foundation, is derived from the global CVE (Common Vulnerabilities and Exposures) system, maintained by MITRE and enriched by the National Vulnerability Database (NVD) at NIST.

Vulnerability Scanning vs. Penetration Testing — A Critical Distinction

This distinction is one of the most commonly misunderstood points in cybersecurity, and understanding it clearly is important both for passing certification exams and for being credible in professional conversations with clients.

Vulnerability scanning is automated and primarily passive in its impact. The scanner discovers and reports weaknesses. It does not prove that those weaknesses are actually exploitable, does not chain vulnerabilities together to achieve a larger goal, and does not demonstrate business impact. A scanner that finds Apache Struts version 2.3.5 will report CVE-2017-5638 (the Equifax breach vulnerability) — but it will not log in to the database, extract customer records, and demonstrate that a breach occurred.

Penetration testing takes vulnerability scan results as starting intelligence and then actively attempts to exploit those vulnerabilities. It confirms which reported vulnerabilities are actually exploitable in the specific environment, chains individual weaknesses into attack paths, pivots from one compromised system to the next, and ultimately produces evidence of what a real attacker could achieve. A penetration tester who finds CVE-2017-5638 will write and execute an exploit, gain a shell on the server, and demonstrate exactly what data is accessible.

The analogy that makes this concrete: vulnerability scanning is like a doctor looking at an X-ray and identifying where there might be fractures. Penetration testing is the doctor pressing on each suspected fracture point to determine which ones are actually broken and how seriously.

Both are essential. Organizations typically run vulnerability scans continuously or weekly, and penetration tests annually or after significant changes. The scans provide broad, frequent coverage; the penetration test provides depth and confirmation of real-world risk.

Where Vulnerability Scanning Sits in the Attack Lifecycle

In the penetration testing methodology, vulnerability scanning comes after active reconnaissance (which built the asset inventory) and directly informs the exploitation phase. The output of a well-executed vulnerability scan is essentially a prioritized shortlist of potential attack vectors, with the most severe and most easily exploitable vulnerabilities at the top.

A professional penetration tester does not run a vulnerability scan, stare at the output, and then immediately start exploiting everything with a CVSS score above 7. The scan output is a starting point — a candidate list that requires human analysis, verification, and strategic thinking before exploitation begins. Which vulnerabilities are actually reachable from the attacker's current position? Which have public exploits available? Which, if exploited, lead toward the engagement's target objectives? Which are likely false positives given the environment? Answering these questions requires the human expertise that automation cannot replace.

3.3.2 How a Typical Automated Vulnerability Scanner Works

The Architecture of a Vulnerability Scanner

Understanding how vulnerability scanners work at a mechanical level helps you interpret their output correctly, understand their limitations, and make better decisions about when and how to use them.

Every major vulnerability scanner — Nessus, OpenVAS, Nexpose, Qualys — shares the same fundamental architecture, even though their implementations differ. The architecture consists of four phases that happen in sequence during every scan.

Phase 1: Discovery and Asset Inventory

Before checking for vulnerabilities, the scanner must know what it is scanning. It performs its own host discovery and port scanning, essentially running a built-in version of what you would do manually with Nmap. It identifies which IP addresses are live, which ports are open, and which services are running. Many scanners use Nmap internally or a similar port scanning engine.

This phase is where the scanner builds its picture of the attack surface. For each discovered host and service, it prepares to execute the relevant vulnerability checks. It would be wasteful to run Apache-specific vulnerability checks against a host that only has Windows services running — the scanner's intelligence about what is running on each host determines which vulnerability checks are relevant.

Phase 2: Fingerprinting and Version Detection

With the open ports identified, the scanner sends protocol-specific probes to determine the exact software and version running on each port. It is doing the same work as Nmap's -sV flag but with a more comprehensive probe database and more sophisticated version extraction logic.

This phase is where the scanner determines that port 8080 is running Apache Tomcat 9.0.37 (not just "something HTTP"), that port 22 is running OpenSSH 8.2p1 on Ubuntu 20.04.5 (not just "something SSH"), and that port 445 is running Windows Server 2016 SMB (not just "Windows SMB"). The more precisely it can determine versions, the more precisely it can match against the vulnerability database.

Phase 3: Vulnerability Checks (Plugins/NVTs)

This is the core of the scanner's work. For each discovered service with a determined version, the scanner consults its vulnerability database and executes the relevant checks. These checks are called "plugins" in Nessus and "Network Vulnerability Tests" (NVTs) in OpenVAS.

Each plugin checks for a specific vulnerability or class of vulnerabilities. Some checks are entirely passive — the scanner simply compares the detected version number against a range of known-vulnerable versions. If Apache Tomcat 9.0.37 is detected, and the vulnerability database knows that versions 9.0.0 through 9.0.43 are vulnerable to CVE-2021-41079, the plugin reports a match.

Other checks are more active — the scanner sends specific probe packets designed to elicit responses that reveal whether a vulnerability exists. For Heartbleed, the scanner sends a specially crafted TLS heartbeat request and measures the response size; a response that is larger than the request indicates the vulnerability is present and the server is returning server memory content. For EternalBlue, the scanner sends a specific SMB negotiation sequence and analyzes the response.

Some checks are behavioral — they attempt to detect vulnerability by observing how the service behaves under specific conditions. These are more reliable than pure version-matching but more likely to cause service disruption.

Phase 4: Reporting and Scoring

With all checks complete, the scanner compiles its findings and assigns severity scores using the CVSS framework. It generates a report that lists each discovered vulnerability with its CVE identifier, CVSS score, affected asset, description of the vulnerability, evidence that the scanner used to determine the vulnerability was present, and remediation guidance.

The quality of this report is where scanners differentiate themselves significantly. A well-configured scanner with authenticated access to the target systems generates reports that are accurate, prioritized, and actionable. A poorly configured scanner running unauthenticated produces reports full of version-matched findings that may or may not be accurate, requiring significant manual verification.

The Plugin/NVT Database — The Scanner's Knowledge Base

The vulnerability database is the scanner's brain. A scanner that has not been updated in six months is a scanner that does not know about the past six months' worth of CVEs — which could be thousands of vulnerabilities. This is not a hypothetical concern: in 2024, NIST's NVD recorded over 29,000 new CVE entries. Keeping scanner databases current is an operational requirement, not an optional maintenance task.

Nessus calls its vulnerability checks "plugins." As of 2024, Nessus has over 200,000 plugins. These plugins are written in a domain-specific language called NASL (Nessus Attack Scripting Language) and distributed through Tenable's update feed. A Nessus subscription includes daily plugin updates — critical for maintaining coverage of newly disclosed vulnerabilities.

OpenVAS uses "Network Vulnerability Tests" (NVTs), also written in NASL. The community feed contains over 160,000 NVTs as of mid-2024. The commercial Greenbone Enterprise feed contains additional tests not available in the community version.

The scanner's plugin/NVT database is essentially a structured library of knowledge about every known vulnerability, organized by affected software, version ranges, and check methodology. When you see Nessus report CVE-2021-44228 (Log4Shell), it is because a plugin exists that knows exactly how to probe for that vulnerability and what a positive response looks like.

3.3.3 The CVE and CVSS Systems — The Language of Vulnerability

CVE — Common Vulnerabilities and Exposures

CVE is the global standard for identifying and naming individual vulnerabilities. Every known vulnerability in publicly released software receives a CVE identifier — a unique reference number that lets security teams, vendors, researchers, and tools speak about a specific vulnerability without ambiguity.

The format is straightforward: CVE-[year]-[sequence number]. CVE-2021-44228 is vulnerability number 44228 discovered and reported in 2021. The year component does not necessarily reflect when the vulnerability was exploited or even when it was first introduced into software — it reflects when the CVE was assigned, which happens when the vulnerability is publicly disclosed or reported to MITRE.

CVE is maintained by MITRE Corporation under sponsorship from the U.S. Department of Homeland Security. The assignment process involves CVE Numbering Authorities (CNAs) — organizations that have been authorized to assign CVE numbers for vulnerabilities in their own products or domains. Microsoft, Google, Apple, Cisco, and hundreds of other organizations are CNAs. When they discover a vulnerability in their own software, they assign it a CVE number as part of their responsible disclosure process.

The National Vulnerability Database (NVD) at NIST enriches CVE records with additional data: CVSS scores, vulnerability classifications (using CWE — Common Weakness Enumeration), links to vendor advisories, and reference URLs. When a security analyst or a vulnerability scanner looks up CVE-2021-44228, they are typically consulting the NVD record for the full picture.

Understanding CVE identifiers is fundamental because they are the common language of vulnerability management. When a client's remediation team asks "which CVEs does this finding address?", when a patch management system tracks patched vs. unpatched CVEs, when a scanner report lists findings by CVE — all of these conversations are only possible because of the CVE system.

Some key CVEs that every cybersecurity professional should know:

CVE-2017-0144 (EternalBlue / MS17-010) — A vulnerability in Windows SMBv1 that allows remote code execution without authentication. This is the vulnerability used by WannaCry ransomware (May 2017) and NotPetya (June 2017). The NSA developed the original exploit, which was leaked by the Shadow Brokers group. Despite Microsoft releasing a patch in March 2017, millions of unpatched machines were compromised.

CVE-2021-44228 (Log4Shell) — A critical vulnerability in Apache Log4j 2, a widely used Java logging library. It allows any input controlled by an attacker that gets logged to trigger JNDI lookups that execute arbitrary code. Because Log4j is embedded in thousands of commercial and open-source applications, the attack surface was enormous. CVSS score: 10.0. Disclosed December 9, 2021; mass exploitation began within hours.

CVE-2014-0160 (Heartbleed) — A buffer over-read vulnerability in OpenSSL's implementation of the TLS heartbeat extension. It allows reading up to 64KB of server memory per request, potentially exposing private keys, session tokens, and user credentials. CVSS score: 7.5. Affected an estimated 17% of all HTTPS servers when disclosed in April 2014.

CVE-2017-5638 (Apache Struts RCE) — A remote code execution vulnerability in Apache Struts 2 (specifically the Jakarta Multipart Parser). Used to breach Equifax in 2017, exposing the personal data of 147 million people. CVSS score: 10.0. A patch was available months before the Equifax breach — the organization simply had not applied it.

CVE-2019-0708 (BlueKeep) — A critical remote code execution vulnerability in Windows Remote Desktop Services. Exploitable without authentication on Windows XP, Vista, 7, Server 2003, and Server 2008. CVSS score: 9.8. Described by Microsoft as "wormable" — capable of spreading automatically between systems without user interaction.

These examples illustrate a recurring pattern: critical vulnerabilities (CVSS 9.0+) that are exploitable without authentication over the network represent the highest-priority findings in any vulnerability assessment.

CVSS — Common Vulnerability Scoring System

CVSS is the framework used to assign severity scores to vulnerabilities. Every vulnerability scanner, every security advisory, and every NVD database record uses CVSS to communicate how severe a vulnerability is. Understanding CVSS deeply is not optional for a cybersecurity professional — it is the vocabulary of severity.

CVSS was developed by the National Infrastructure Advisory Council (NIAC) and is now maintained by FIRST (Forum of Incident Response and Security Teams). The current version in widespread use is CVSS v3.1 (released June 2019), with CVSS v4.0 released in November 2023 beginning to see adoption.

The Three Score Groups

CVSS produces not one score but three distinct scores, each serving a different purpose.

The Base Score represents the intrinsic characteristics of the vulnerability — how it behaves, how exploitable it is, and what impact successful exploitation has. The Base Score does not change based on time, environment, or context. It is calculated by the vendor or security researcher when the vulnerability is disclosed and serves as the universal severity anchor. When people say "this vulnerability has a CVSS score of 9.8," they are almost always referring to the Base Score.

The Temporal Score modifies the Base Score based on factors that change over time. Has a public exploit been released? Is a patch available? How confident is the security community that this vulnerability actually exists? A vulnerability might start with a Base Score of 9.8 but have a lower Temporal Score initially because no public exploit exists and the vulnerability is only theoretically confirmed. As exploit code appears and exploitation in the wild is confirmed, the Temporal Score approaches the Base Score. This score helps organizations understand urgency — a vulnerability with confirmed active exploitation is more urgent than one that is only theoretically exploitable.

The Environmental Score allows an organization to customize the score based on their specific environment. A vulnerability in a web server component might have a high Base Score for confidentiality impact — but if your organization does not store sensitive data on that server, the actual confidentiality risk to your organization is lower. The Environmental Score lets you reflect this reality in your risk prioritization.

The Base Score Metrics — Understanding Each Component

The Base Score is calculated from eight metrics, organized into Exploitability metrics (how an attacker uses the vulnerability) and Impact metrics (what happens when exploitation succeeds).

Attack Vector (AV) describes how the attacker reaches the vulnerable component. Network (AV:N) means the attacker can exploit it from anywhere on the internet — the highest severity. Adjacent (AV:A) means the attacker must be on the same local network segment. Local (AV:L) means the attacker needs a local shell account or physical access. Physical (AV:P) means physical access to the hardware is required — the lowest severity. A remote code execution vulnerability in a publicly accessible web server is AV:N. A privilege escalation vulnerability that requires a local shell first is AV:L.

Attack Complexity (AC) describes conditions outside the attacker's control that must exist for exploitation to succeed. Low (AC:L) means the vulnerability can be exploited reliably on any vulnerable system — no special conditions, no timing requirements. High (AC:H) means exploitation requires circumstances that cannot be guaranteed, such as a race condition that requires precise timing, or a man-in-the-middle position that must be established first. The difference between AC:L and AC:H can significantly affect a vulnerability's practical exploitability — a race condition with a 1-in-1000 success rate is very different from a vulnerability exploitable 100% of the time.

Privileges Required (PR) describes the level of access the attacker needs before exploitation. None (PR:N) means no authentication or account is needed — the attacker can exploit directly without any prior access. Low (PR:L) means a standard user account is needed. High (PR:H) means an administrator account is needed. PR:N vulnerabilities — those exploitable without any authentication — are the most dangerous class because they require no prior compromise.

User Interaction (UI) describes whether exploitation requires action from a user other than the attacker. None (UI:N) means the attacker can exploit autonomously without any victim involvement. Required (UI:R) means a user must take an action — click a link, open a file, visit a page — for exploitation to succeed. Vulnerabilities with UI:N are more dangerous because they can be exploited at scale without requiring any social engineering or victim cooperation.

Scope (S) is one of the subtler CVSS concepts. It addresses whether a vulnerability in one component can impact resources in a different security domain. Unchanged (S:U) means exploitation only affects the vulnerable component itself. Changed (S:C) means exploitation allows impact to resources governed by a different security authority — for example, a vulnerability in a web application sandbox that allows escaping the sandbox and affecting the underlying operating system. A scope change dramatically increases a vulnerability's severity because it enables lateral movement and privilege escalation beyond the initially compromised component.

Confidentiality Impact (C), Integrity Impact (I), and Availability Impact (A) each measure the impact on the corresponding security property if the vulnerability is successfully exploited. Each is scored as None (no impact), Low (some impact with limited scope), or High (total loss — complete data exposure, complete data modification, or complete denial of service). A vulnerability that results in complete data loss on all aspects (C:H, I:H, A:H) is the most severe from an impact perspective.

Reading a CVSS Vector String

CVSS scores are accompanied by a vector string that encodes all the metric values in a compact, standardized format. The vector string allows you to understand exactly why a vulnerability received its score.

For CVE-2021-44228 (Log4Shell), the CVSS v3.1 vector is:
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H

Reading each component: Network attack vector (reachable from the internet), Low complexity (reliably exploitable), No privileges required (anonymous exploitation), No user interaction needed, Scope Changed (sandbox escape enabling OS-level impact), and High impact on all three CIA properties. This is the most dangerous possible combination of metrics — and it produced a score of 10.0, the maximum.

Compare this to a hypothetical local privilege escalation:
CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H

This requires local access (AV:L) and a standard user account (PR:L), but once those prerequisites are met, it is reliable (AC:L), requires no victim interaction (UI:N), does not escape its security scope (S:U), and results in full system compromise (C:H/I:H/A:H). Score: approximately 7.8 — High, but significantly less critical than Log4Shell because an attacker must already have local access.

Severity Ranges — How to Use CVSS Scores for Prioritization

CVSS Score Range	Severity Label	Typical Priority
9.0 – 10.0	Critical	Immediate — patch within 24–48 hours; investigate exploitation immediately
7.0 – 8.9	High	Urgent — patch within 7–14 days
4.0 – 6.9	Medium	Important — patch within 30 days
0.1 – 3.9	Low	Plan — patch in next maintenance cycle
0.0	None / Informational	Track — no action required

These timelines represent reasonable targets, not universal rules. An organization processing payment card data (PCI DSS scope) might have more aggressive patching requirements. A critical infrastructure organization might apply different risk weighting to availability versus confidentiality. The CVSS score is an input to prioritization, not a complete prioritization decision on its own.

CVSS v4.0 — What Changed in the Latest Version

CVSS v4.0, released November 2023, introduced several improvements for completeness:

The Temporal metrics were renamed to "Threat Metrics" with a cleaner structure. A new metric called "Attack Requirements" was added to complement "Attack Complexity" — separating conditions about the environment from conditions about attack execution. Supplemental metrics were introduced to provide additional context without affecting the score: Automatable (can this be weaponized at scale?), Recovery (how hard is it to restore after exploitation?), Safety (does this affect human safety?), and Value Density (how much valuable data is in the impacted component?).

Most current tooling and CVE records still use CVSS v3.1, but CVSS v4.0 adoption is growing and will become the standard over the next few years.

3.3.4 Types of Vulnerability Scans

Not All Scans Are the Same — Choosing the Right Approach

Just as there are different types of nmap scans suited to different situations, there are multiple types of vulnerability scans, each with different levels of access, different levels of accuracy, and different operational implications. Choosing the wrong scan type leads to either inaccurate results or unnecessary operational risk.

Unauthenticated (External / Black Box) Scanning

An unauthenticated scan runs without providing any credentials to the target systems. The scanner operates purely from the network perspective — it can see whatever an external, unprivileged attacker could see. It sends probes over the network, receives responses, and draws conclusions from what those responses reveal.

Think of this as a security inspector walking around the outside of a building, looking through windows, testing doors from the outside, checking whether the locks look functional — but never going inside. The inspector can identify quite a lot from this external perspective: broken windows, unlocked doors, outdated security notices on the door, visitors going in and out. But they cannot see the filing cabinets inside, check whether confidential documents are properly secured, or audit the internal access control system.

What unauthenticated scans find well:

Services listening on open ports (web servers, SSH, FTP, databases exposed to the network)
Version-based vulnerabilities in services that announce their version in banners or response headers
Default credentials on network services (the scanner can attempt to authenticate with known defaults)
Protocol-level vulnerabilities (TLS weaknesses, SMB version vulnerabilities, DNS misconfigurations)
Network-level misconfigurations (open ports that should be closed, unnecessary services)
Web application vulnerabilities accessible without authentication (publicly exposed admin panels, login page bypasses)

What unauthenticated scans miss:

Vulnerabilities in software that does not expose version information over the network
Unpatched software on workstations and servers (patch level requires authenticated access to check)
Configuration weaknesses inside the operating system (registry settings, file permissions, service configurations)
Installed software with known vulnerabilities that does not run a network service
Local privilege escalation vulnerabilities
Database access control weaknesses (requires database credentials)

The vulnerabilities found by unauthenticated scans tend to be the most severe from a network security perspective — they are accessible to anyone who can reach the network. In many ways, these are the most critical findings because they represent zero-barrier-to-entry attack paths.

Authenticated (Credentialed / Internal) Scanning

An authenticated scan provides the scanner with valid credentials — a username and password (or SSH key, or Windows domain account, or database credentials) that allow it to log in to each target system and perform checks from within the authenticated session.

To extend the building inspection analogy: the inspector now has a master key. They can open every door, examine every filing cabinet, check every room's configuration, and audit the complete internal state of the building. The inspection is far more thorough.

What authenticated scans find that unauthenticated scans miss:

The complete list of installed software and their versions (cross-referenced against CVE database)
Missing patches across the entire installed software inventory
Operating system misconfigurations (overly permissive file permissions, insecure registry settings, unnecessary services running)
User account issues (disabled accounts with active sessions, accounts with never-expiring passwords, local administrators)
Password policy compliance (password history enforcement, lockout threshold, complexity requirements)
Configuration compliance (CIS Benchmarks, DISA STIGs, PCI DSS controls)
Database security settings accessible only with database credentials
Application configuration weaknesses not visible from the network

The tradeoff: authenticated scanning requires obtaining and securely managing credentials for every system to be scanned. In large environments, this is a significant coordination effort. It also requires ensuring the scanner's account has sufficient privileges to perform the checks — typically a domain administrator account for Windows environments, or root/sudo access for Linux.

Credential storage and security for authenticated scanning: The credentials used for scanning must be protected as carefully as any privileged account. A compromised credential store that contains domain admin credentials for hundreds of systems is a catastrophic security incident. Enterprise vulnerability management platforms use encrypted credential vaults, require authentication before credential retrieval, and maintain audit logs of every use.

Network Scanning

Network scanning focuses on the network infrastructure layer — the services, ports, and protocols visible across the network. This is what most people mean when they say "vulnerability scan" in a general context. The scanner probes network services, identifies versions, and checks for known network-level vulnerabilities.

Network scanning is the foundation of most vulnerability assessment programs. It covers web servers, mail servers, SSH services, database listeners, file sharing services, and any other network-accessible service. It is the type of scan most directly relevant to penetration testing reconnaissance because it identifies the external attack surface.

Web Application Scanning

Web application scanning is a specialized discipline focused specifically on vulnerabilities in web applications — not the web server infrastructure (Apache, nginx, IIS) but the application running on top of it.

A web application scanner does not just probe ports and check service versions. It crawls the application, discovering all pages, forms, parameters, and API endpoints. For each discovered input point, it sends specially crafted payloads designed to trigger specific vulnerability classes: SQL injection payloads that attempt to manipulate database queries, Cross-Site Scripting (XSS) payloads that attempt to inject malicious JavaScript into page output, path traversal payloads that attempt to read files outside the web root, command injection payloads that attempt to execute operating system commands.

Web application vulnerabilities are classified by the OWASP Top 10 — a regularly updated list of the most critical web application security risks (covered extensively in Module 6). The primary web application scanners are:

OWASP ZAP (Zed Attack Proxy) — Free and open-source. The most widely used web application scanner for penetration testing. Has both automated scanning and manual testing proxy capabilities.

Burp Suite — The industry standard commercial tool (with a free Community edition). More sophisticated than ZAP for manual testing, with powerful active scanner capabilities in the Professional edition.

Nikto — A simpler, faster web server scanner focused on common misconfigurations and known dangerous files, not deep application-level testing.

Agent-Based Scanning

Agent-based scanning involves installing a lightweight software agent on each managed endpoint. The agent performs vulnerability checks locally and reports findings to a central management platform.

The advantage over traditional network scanning is accuracy: the agent has complete, authenticated access to the system from within. It can see every installed software package, every running process, every configuration file. There are no network-based detection limitations, no issues with services that hide their version information, no problems with firewall rules that block scanner traffic.

The disadvantage is the requirement to deploy and maintain agents across every endpoint — which in large enterprises with thousands of endpoints is a significant operational commitment. Agentless scanning (traditional network scanning with credentials) is simpler to manage but less thorough.

Enterprise platforms like Tenable.sc, Qualys VMDR, and Rapid7 InsightVM support both agent-based and agentless scanning, often using a hybrid approach where agents are deployed on workstations and laptops (which are frequently disconnected from the corporate network) while servers are scanned agentlessly over the network.

Compliance Scanning

Compliance scanning evaluates systems against specific security benchmarks and regulatory requirements, rather than (or in addition to) looking for CVE-based vulnerabilities.

The CIS Benchmarks (Center for Internet Security) are the most widely referenced configuration standards. There are CIS Benchmarks for Windows operating systems, Linux distributions, major cloud platforms, databases, web servers, and network devices. Each benchmark contains hundreds of specific configuration checks — whether audit logging is enabled, whether the firewall is configured correctly, whether specific registry values are set appropriately, whether unused services are disabled.

PCI DSS compliance scanning verifies that payment card data environments meet all applicable PCI DSS requirements. HIPAA compliance scanning checks for HIPAA Security Rule requirements. DISA STIGs (Defense Information Systems Agency Security Technical Implementation Guides) are used for U.S. federal government systems.

Compliance scan results are typically expressed as "pass/fail" against each control, rather than CVSS scores. The output is a compliance percentage and a list of failing controls with remediation guidance — precisely the format needed for compliance reporting.

3.3.5 The Major Vulnerability Scanning Tools — Deep Comparison

Nessus (Tenable)

What it is: Nessus, developed by Tenable, is the most widely deployed commercial vulnerability scanner in the world. Since its initial release in 1998, it has become the industry benchmark — the scanner most frequently mentioned in job listings, most commonly encountered in enterprise environments, and most referenced in certification curricula.

The versions:

Nessus Essentials is the free tier, limited to 16 IP addresses. It uses the same interface as the professional versions and is the appropriate starting point for students and home lab users. It is genuinely Nessus, not a crippled demo — the scan quality is identical, just the IP count is limited.

Nessus Professional is the paid single-scanner product used by individual consultants and small security teams. As of 2024, it costs approximately $3,990 per year. It provides unlimited IP scanning, advanced reporting, and the full plugin library.

Tenable.sc (formerly SecurityCenter) is the enterprise platform for large organizations running multiple Nessus scanners. It provides centralized management, dashboards, trend tracking, and role-based access control across many scanners and many business units.

Tenable.io is Tenable's cloud-based platform, which adds continuous monitoring, agent-based scanning, web application scanning, and cloud infrastructure scanning in a unified SaaS platform.

Why Nessus is the standard: Nessus has the largest plugin library (200,000+), the most mature update cycle for new vulnerability checks, and the most polished interface for report generation. It is also the scanner most commonly found in large enterprise environments, which means knowing Nessus is directly career-relevant. Many job listings for vulnerability management roles list Nessus experience as a requirement.

Its limitations: The primary limitation is cost — Nessus Professional is expensive for individual use. Detection accuracy has also been scrutinized: a 2024 benchmark study found that Nessus detects for a larger percentage of known vulnerabilities than it successfully identifies in practice, suggesting some detection gaps between plugin availability and actual detection capability. False positives exist in every scanner, but Nessus's version-matching approach can produce findings for software that has been patched at the package level even though the version number did not change.

OpenVAS / Greenbone (GVM)

What it is: OpenVAS (Open Vulnerability Assessment System) is the most significant free, open-source vulnerability scanner. It originated as a fork of the Nessus codebase in 2005 when Tenable closed Nessus's source code. Since then, it has been maintained and developed independently. Today, OpenVAS is the scanning engine at the heart of the Greenbone Vulnerability Management (GVM) platform, maintained by Greenbone Networks.

The architecture: OpenVAS does not stand alone — it is one component in the GVM stack. The full stack consists of the OpenVAS Scanner daemon (ospd-openvas) which runs the actual checks, the Greenbone Vulnerability Manager (GVM) API layer which manages scans and stores results, and the Greenbone Security Assistant (GSA) web interface which provides the user-facing interface. For beginners, "OpenVAS" is often used to refer to the entire GVM stack.

The community vs. commercial feed: The Greenbone Community Feed is free and contains over 160,000 NVTs (Network Vulnerability Tests). The Greenbone Enterprise feed (subscription) contains additional tests, including more coverage of enterprise technologies and compliance checks. For learning and lab use, the community feed is comprehensive.

Why OpenVAS matters: It is free. For security students, independent consultants, and organizations with budget constraints, this is decisive. The scan quality for the most critical, high-CVSS vulnerabilities is comparable to commercial scanners. A 2024 analysis found OpenVAS leads commercial scanners in remote check coverage for medium-severity CVEs, while Nessus Professional has broader coverage for the most critical remote vulnerabilities. The gap is meaningful but not absolute.

Installation and setup: OpenVAS requires more setup than Nessus. On Kali Linux, the installation is managed through the package manager, but the initial feed synchronization (downloading all 160,000+ NVT definitions) takes significant time and the GVM stack has dependency requirements that need careful management. The effort is worthwhile for a learning environment.

# Installation on Kali Linux
sudo apt update && sudo apt install -y gvm

# Initial setup (downloads feeds, creates users, configures certificates)
# This takes 15-30 minutes on the first run
sudo gvm-setup

# Start GVM services
sudo gvm-start

# Access the web interface
# Opens at: https://127.0.0.1:9392
# Default credentials are shown during setup (generated randomly)

# Check if everything is running correctly
sudo gvm-check-setup

# Update the NVT feed (run regularly — daily ideally)
sudo greenbone-nvt-sync
sudo greenbone-feed-sync --type SCAP
sudo greenbone-feed-sync --type CERT

Nuclei (ProjectDiscovery)

What it is: Nuclei is a modern, template-based vulnerability scanner developed by ProjectDiscovery. Unlike Nessus and OpenVAS which use proprietary plugin/NVT systems, Nuclei uses YAML-formatted templates that define vulnerability checks in a simple, readable format. The template library is maintained as a public GitHub repository with thousands of community-contributed templates, and adding new vulnerability checks is as simple as writing a YAML file.

Why Nuclei is rapidly growing in importance: Nuclei excels at a different scan category than OpenVAS or Nessus. While the traditional scanners focus on infrastructure-level vulnerabilities (CVE-based version matching, OS configuration checks), Nuclei specializes in web application and API vulnerability detection, subdomain takeover checks, exposure detection (publicly accessible sensitive files, admin panels, backup files), and CVE-specific probe-based checks.

Its speed is exceptional — Nuclei is designed for high-concurrency scanning and can scan thousands of targets rapidly. For bug bounty hunters, red teamers, and penetration testers dealing with large external attack surfaces, Nuclei has become a standard part of the workflow.

The template ecosystem: The official Nuclei template library contains thousands of templates organized by category: CVEs, exposures, misconfiguration, technologies, default-logins, takeovers, and more. When a new CVE is disclosed, community members often publish Nuclei templates within hours — sometimes before commercial scanners have updated their plugin databases.

# Install Nuclei
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest

# Or on Kali Linux
sudo apt install nuclei

# Update templates (run regularly)
nuclei -update-templates

# Basic scan against a target
nuclei -u https://target.com

# Scan with specific template categories
nuclei -u https://target.com -tags cve
nuclei -u https://target.com -tags exposure
nuclei -u https://target.com -tags default-login

# Scan with specific severity levels
nuclei -u https://target.com -severity critical,high

# Scan a list of targets
nuclei -list targets.txt -tags cve -severity critical

# Scan for a specific CVE
nuclei -u https://target.com -id CVE-2021-44228

# Output to file
nuclei -u https://target.com -o nuclei_results.txt -json

Nikto

What it is: Nikto is a free, open-source web server scanner with a narrow but well-executed purpose: it scans web servers specifically for known dangerous files and programs, outdated server software and components, and server configuration misconfigurations.

Nikto is not a deep application-layer scanner in the way Burp Suite or OWASP ZAP are. It does not test for SQL injection by trying injection payloads in form fields. What it does is check for thousands of specific files and paths that should not be publicly accessible — backup files, configuration files, test scripts, old CMS installations, and similar — and report any that respond with content rather than a 404.

Nikto also checks HTTP response headers for security misconfigurations: missing security headers like Content-Security-Policy, X-Frame-Options, and Strict-Transport-Security; server headers that unnecessarily disclose version information; and cookie security flags.

For a penetration tester, Nikto is a quick first pass against any discovered web server — run it early, get a fast overview of low-hanging fruit and obvious misconfigurations, then move to deeper tools.

# Basic Nikto scan
nikto -h http://target.com
nikto -h https://target.com

# Scan specific port
nikto -h http://target.com -p 8080

# With SSL
nikto -h https://target.com -ssl

# Verbose output
nikto -h http://target.com -Display V

# Save output
nikto -h http://target.com -o nikto_results.html -Format htm
nikto -h http://target.com -o nikto_results.xml -Format xml

# Specify specific tests to run
nikto -h http://target.com -Tuning 9   # Run SQL injection tests
nikto -h http://target.com -Tuning 4   # Run XSS tests

# Use with proxy (for interception in Burp Suite)
nikto -h http://target.com -useproxy http://127.0.0.1:8080

Qualys VMDR and Other Enterprise Platforms

Qualys Vulnerability Management, Detection and Response (VMDR) is one of the leading cloud-based enterprise vulnerability management platforms. Unlike Nessus (which runs as a local scanner) or OpenVAS (which is self-hosted), Qualys is delivered entirely as a cloud service. Organizations deploy lightweight Qualys Cloud Agents on managed endpoints and Qualys Virtual Scanners in internal network segments.

Qualys is mentioned in this context because it is extremely common in large enterprise environments and frequently referenced in job listings for vulnerability management roles. If you interview for an enterprise security position, Qualys experience may be listed as a requirement. Understanding what it does conceptually — cloud-based, agent-supported, continuous scanning with compliance policy checking and asset discovery — is relevant professional knowledge even if hands-on access requires a subscription.

Other enterprise platforms worth knowing by name: Rapid7 Nexpose / InsightVM (similar positioning to Nessus Professional/Tenable.sc), Microsoft Defender Vulnerability Management (for organizations standardized on Microsoft security stack), and CrowdStrike Falcon Spotlight (agent-based vulnerability management integrated with endpoint detection and response).

3.3.6 Vulnerability Scanning with Kali Tools — Practical Reference

Setting Up and Running a Professional Scan Workflow

In a penetration testing engagement, vulnerability scanning is not a single-click operation. It is a deliberate, staged process that uses multiple tools targeting different aspects of the attack surface. The workflow presented here is representative of how professional penetration testers approach vulnerability scanning in real engagements.

Phase 1 — Infrastructure Vulnerability Scanning with OpenVAS/GVM

For infrastructure-level vulnerability assessment (servers, network devices, workstations), OpenVAS is the primary tool on Kali. The process involves creating a scan target, selecting a scan configuration, running the scan, and analyzing results.

Creating a scan in the GVM web interface (https://127.0.0.1:9392):

Navigate to Scans → Tasks → New Task. Provide a name for the task. Under "Scan Targets," create a new target specifying the IP addresses or CIDR range to scan. Select the scan configuration — typically "Full and Fast" for a comprehensive authenticated scan or "Discovery" for a quick initial overview. If performing an authenticated scan, add credentials in the Credentials section before creating the task.

For penetration testing contexts, the "Full and Fast" configuration runs all applicable NVTs with optimized timing. The "Full and Very Deep" configuration runs more thorough checks including some that may be disruptive to services — use this with caution on production systems.

From the command line using gvm-cli:

# List available scan configurations
gvm-cli socket --gmp-username admin --gmp-password [pass] \
  --xml "<get_configs/>"

# Create a target (replace with actual IP/range)
gvm-cli socket --gmp-username admin --gmp-password [pass] \
  --xml "<create_target><name>PenTest Target</name><hosts>10.10.10.0/24</hosts></create_target>"

# Check scan status
gvm-cli socket --gmp-username admin --gmp-password [pass] \
  --xml "<get_tasks/>"

Phase 2 — Web Application Scanning with Nikto

For every web server discovered during port scanning, run Nikto as a fast initial check before deeper application testing.

# Quick scan against all discovered web servers
# Assuming live_web_servers.txt contains one IP:port per line
while IFS= read -r target; do
    echo "[*] Scanning $target"
    nikto -h "http://$target" -o "nikto_${target//[:\/]/_}.txt" -Format txt
done < live_web_servers.txt

# Common Nikto findings worth noting:
# - "Server leaks inodes via ETags" — information disclosure
# - "The anti-clickjacking X-Frame-Options header is not present" — missing security header
# - "Retrieved x-powered-by header: PHP/7.4.3" — version disclosure
# - "Allowed HTTP Methods: GET, POST, OPTIONS, DELETE" — dangerous HTTP methods enabled
# - "Default account found for 'admin'" — default credential finding
# - "OSVDB-XXXX: /phpmyadmin/: phpMyAdmin directory found" — exposed admin interface

Phase 3 — CVE-Specific Scanning with Nuclei

For targeted CVE checks, especially on external-facing web services, Nuclei provides fast, accurate results.

# Scan for critical CVEs specifically
nuclei -l live_hosts.txt -tags cve -severity critical -o nuclei_critical.txt

# Scan for exposed sensitive files and panels
nuclei -l web_servers.txt -tags exposure -o nuclei_exposures.txt

# Scan for default credentials
nuclei -l web_servers.txt -tags default-login -o nuclei_defaultcreds.txt

# Check for specific high-profile CVEs
nuclei -l targets.txt -id CVE-2021-44228  # Log4Shell
nuclei -l targets.txt -id CVE-2021-26855  # ProxyLogon (Exchange)
nuclei -l targets.txt -id CVE-2022-22965  # Spring4Shell

# Full combined scan with output
nuclei -l targets.txt \
       -severity critical,high \
       -tags cve,exposure,default-login \
       -o nuclei_results.json \
       -json

Phase 4 — Nmap NSE Vulnerability Scripts

Nmap's scripting engine provides targeted vulnerability checks that integrate naturally with the existing scan workflow.

# Run all vulnerability category scripts against discovered hosts
sudo nmap --script vuln -p 21,22,23,25,80,443,445,3389 \
          -iL live_hosts.txt \
          -oA nmap_vuln_scan

# Specific high-value vulnerability checks:

# EternalBlue (MS17-010) — WannaCry/NotPetya vulnerability
sudo nmap --script smb-vuln-ms17-010 -p 445 10.10.10.0/24

# BlueKeep (CVE-2019-0708) — RDP vulnerability
sudo nmap --script rdp-vuln-ms12-020 -p 3389 10.10.10.0/24

# Heartbleed (CVE-2014-0160) — OpenSSL vulnerability  
sudo nmap --script ssl-heartbleed -p 443,8443 10.10.10.0/24

# SSL/TLS cipher weaknesses
sudo nmap --script ssl-enum-ciphers -p 443 10.10.10.0/24

# SMB vulnerability comprehensive check
sudo nmap --script "smb-vuln-*" -p 445 10.10.10.0/24

# Apache Struts (CVE-2017-5638) — Equifax breach vulnerability
sudo nmap --script http-vuln-cve2017-5638 -p 80,443,8080 10.10.10.0/24

# HTTP server methods check (dangerous methods like PUT, DELETE)
sudo nmap --script http-methods -p 80,443,8080 10.10.10.0/24

# Default HTTP credentials
sudo nmap --script http-default-accounts -p 80,443 10.10.10.0/24

# Database vulnerability checks
sudo nmap --script mysql-vuln-cve2012-2122 -p 3306 10.10.10.0/24
sudo nmap --script ms-sql-empty-password -p 1433 10.10.10.0/24

Phase 5 — Searchsploit for Exploit Verification

After identifying vulnerable versions, searchsploit (the command-line interface to Exploit-DB) helps quickly determine whether public exploits exist.

# Install/update exploit database
sudo searchsploit --update

# Search for exploits by software name and version
searchsploit apache tomcat 9.0.37
searchsploit openssh 7.4
searchsploit "windows server 2016"
searchsploit log4j

# Search for specific CVE
searchsploit CVE-2021-44228
searchsploit CVE-2017-0144

# Get exploit details
searchsploit -x 47837   # View exploit by ID
searchsploit -p 47837   # Show path to exploit file
searchsploit -m 47837   # Copy exploit to current directory

# Output as JSON for automated processing
searchsploit --json apache tomcat | python3 -m json.tool

3.3.7 Challenges to Consider When Running a Vulnerability Scan

The Realities of Professional Vulnerability Scanning

A significant portion of what separates junior security practitioners from senior ones is their understanding of vulnerability scanner limitations. Scanners are powerful tools, but they produce imperfect output that requires skilled human interpretation. Understanding the challenges inherent in vulnerability scanning is essential for delivering accurate penetration test reports and credible vulnerability assessments.

Challenge 1: False Positives — The Scanner Cried Wolf

A false positive is when the scanner reports that a vulnerability exists when it actually does not. This is the most common type of scanner inaccuracy and represents a genuine operational challenge.

Consider this scenario: a scanner detects that Apache HTTP Server version 2.4.41 is running and reports it as vulnerable to CVE-2021-41773 (path traversal vulnerability affecting Apache 2.4.49). But the scanner's version detection was wrong — the actual version running is 2.4.51, which is patched. The CVE is reported, a ticket is raised, a developer spends two hours investigating, and ultimately concludes the report was wrong. That wasted time is the operational cost of a false positive.

False positives arise from several sources:

Version mismatch errors occur when the scanner cannot precisely determine the software version. If a service has been configured to hide its version number (a common security hardening practice), the scanner may assume the version is the most recently detected one or use heuristics that produce incorrect results.

Backported patches are a particularly common cause of false positives in Linux distributions. Enterprise Linux distributions like Red Hat Enterprise Linux and Ubuntu LTS frequently backport security patches to older version branches rather than upgrading to the latest version. This means a system running Apache 2.4.38 might have all the patches from 2.4.51 backported into it — but the version number still reads 2.4.38. A scanner that only checks version numbers will report all vulnerabilities affecting versions 2.4.39 through 2.4.50 as present, when in fact the patches have been applied at the package level.

Mitigation controls can render a vulnerability unexploitable without patching. A vulnerability that requires a specific module to be enabled might be reported as present even on a system where that module is disabled. The vulnerability technically exists in the software, but it is not exploitable in the current configuration.

How professionals handle false positives: Every significant finding from an automated scan should be manually verified before being included in a penetration test report. Manual verification might involve checking the package manager's changelog to confirm backported patches, attempting to reproduce the exploit behavior, or consulting the vendor advisory to understand the precise conditions required for exploitation. A finding that cannot be manually confirmed should be noted as "unverified" or "potential false positive" in the report.

Challenge 2: False Negatives — The Scanner Missed the Elephant in the Room

A false negative is when a real vulnerability exists but the scanner fails to report it. This is arguably more dangerous than a false positive, because false negatives create a false sense of security — the team believes the system is clean when it is not.

False negatives occur for several reasons:

Unknown vulnerabilities (zero-days) are by definition not in any scanner database. A vulnerability that has not yet been publicly disclosed cannot be detected by signature-based scanning. This is a fundamental limitation of vulnerability scanning that cannot be solved within the scanning paradigm.

Custom or proprietary software is not covered by scanner plugins. If an organization has developed in-house applications, the vulnerability scanner knows nothing about their specific weaknesses. The scanner might detect that the application is running and what HTTP framework it uses, but it cannot know about SQL injection vulnerabilities in the application's custom query logic or authentication bypass vulnerabilities in its login implementation.

Logic vulnerabilities — flaws in business logic rather than software implementation — are invisible to automated scanners. An e-commerce application that can be exploited to purchase items at a negative price, or an authentication system that can be bypassed by manipulating request parameters, requires human analysis to discover.

Deeply buried vulnerabilities may require a chain of conditions that the scanner never tests. A vulnerability that is only reachable after authenticating as a specific user type, navigating to a specific page, and submitting a specific form may never be reached by a scanner's automated crawl.

Configuration-dependent vulnerabilities might not be triggered by generic scanner probes. Some vulnerabilities only manifest under specific configuration states or runtime conditions that a scanner's probe sequence does not create.

This is why penetration testing — with its human analysis, creative thinking, and ability to chain multiple techniques together — is irreplaceable even for organizations that run comprehensive automated vulnerability scanning. The scanner finds what it knows to look for. The penetration tester finds everything the scanner missed.

Challenge 3: Scan Impact on Production Systems

Running vulnerability scans against production systems carries inherent operational risk. Scanners send large volumes of probes in short periods of time. Some of those probes test for vulnerabilities by sending intentionally malformed or unexpected input. This traffic can:

Consume network bandwidth to a degree that impacts legitimate users. A full scan of a /16 network with hundreds of thousands of ports can generate gigabits of traffic.

Trigger application errors when scanner probes hit error-handling code paths that are not tested under normal operations. An application might process millions of normal requests flawlessly, but break when it receives a scanner's SQL injection test payload.

Exhaust database connection pools. Scanners that rapidly open and close many simultaneous connections to database services can consume all available connection slots, causing legitimate application database connections to fail.

Trigger IDS/IPS blocking rules that block legitimate traffic. An IDS that detects a vulnerability scan pattern might block the scanner's source IP — but if that source IP is your legitimate testing machine, all legitimate access from that IP gets blocked.

Crash vulnerable services. Some vulnerability checks — particularly denial-of-service checks and buffer overflow detection — inherently risk crashing the service they are testing. Running these against a production web server during business hours could cause a service outage.

Professional mitigations:

Run scans during designated maintenance windows when the business impact of potential disruption is minimized. Coordinate with operations teams so they know scanning is occurring and can distinguish scan-related alerts from real incidents. Start with non-intrusive scan configurations and escalate to more invasive checks only after verifying that initial scans did not cause disruption. Exclude safety-critical systems or systems where any disruption is unacceptable from scanning scope.

Challenge 4: Scope and Coverage Gaps

A vulnerability scanner can only scan what it knows about. This creates a coverage gap that the term "shadow IT" describes: systems, applications, and services that exist on the network but are not formally documented or included in the official asset inventory.

A developer spins up a test server to prototype a new feature. They use a cloud provider account with a personal credit card. The server runs for six months, gets forgotten, and now sits with an unpatched OS and no monitoring — completely invisible to the organization's vulnerability management program.

A network printer is connected to the corporate network. Its embedded web interface runs outdated firmware. Nobody thinks to include it in vulnerability scans because "it's just a printer."

A contractor installs a remote access tool on a workstation so they can support a client. The tool is not approved by IT, creates a backdoor-like access path, and is never removed when the contract ends.

None of these appear in a vulnerability scan unless the scan covers the full IP range where these devices exist. This is why penetration testing pairs host discovery (active reconnaissance to find all live hosts) with vulnerability scanning — you cannot scan what you do not know exists.

Challenge 5: Keeping Scanner Databases Current

The CVE ecosystem produces thousands of new vulnerability disclosures every month. A vulnerability scanner that has not been updated in 30 days is already missing potentially hundreds of new CVEs. A scanner that has not been updated in six months might miss thousands.

This is not theoretical. In 2021, Log4Shell (CVE-2021-44228) was disclosed on December 9. Organizations that had not updated their vulnerability scanners since December 8 had scanner databases that did not include any Log4Shell check. Mass exploitation began within 24 hours of disclosure — meaning organizations needed to scan for Log4Shell immediately, not after their next scheduled quarterly scanner update.

Professional vulnerability management programs update scanner databases daily and often multiple times per day for critical disclosures. Some organizations also maintain subscriptions to vulnerability intelligence services (like Tenable's Vulnerability Priority Rating or Qualys TruRisk) that provide additional context about exploitation likelihood and attack trends.

Challenge 6: Authenticated vs. Unauthenticated Accuracy Trade-offs

As discussed in the scan types section, authenticated and unauthenticated scans produce different results with different reliability characteristics. In a penetration testing context, there is an additional layer of complexity: the engagement might not authorize providing credentials to the scanner.

A penetration test that simulates an external attacker would not provide credentials — because an external attacker has none. The scan results will reflect what is visible without authentication. This is actually the most valuable perspective for the client: it shows exactly what an attacker on the internet could identify and potentially exploit without any prior access.

But for a comprehensive vulnerability assessment — for example, a PCI DSS assessment that must identify all vulnerabilities affecting the Cardholder Data Environment — authenticated scanning is required to get complete coverage. The choice of scan type must align with the engagement objectives.

3.3.8 Vulnerability Scanning in the Penetration Testing Lifecycle

Where Scanning Fits and How to Use Results

Vulnerability scanning in penetration testing serves as a bridge between reconnaissance (knowing what exists) and exploitation (proving vulnerabilities are real and impactful). Understanding exactly how to use scan results to drive the next phase is a skill that distinguishes effective penetration testers from those who just run tools.

Using Scan Results to Prioritize Exploitation Targets

After receiving vulnerability scan results, the penetration tester does not immediately begin exploiting every critical finding. Instead, they analyze the results through several lenses:

Exploitability from current position: A critical vulnerability in a database server is only useful if the attacker can reach it. If the database is on an internal network segment not directly reachable from the external network, it cannot be the first target — it becomes a target for after lateral movement. The penetration tester maps findings to reachability from their current access position.

Public exploit availability: A critical vulnerability with a CVSS score of 9.8 is very concerning — but if no public exploit exists for it, exploitation requires developing a custom exploit, which is time-consuming and high-risk. A vulnerability with a CVSS score of 7.5 but a well-documented, freely available Metasploit module is often a more practical exploitation target. Searching searchsploit and Exploit-DB for each finding reveals exploit availability.

Alignment with engagement objectives: What is the penetration test trying to demonstrate? If the objective is to reach the financial database and exfiltrate a sample record, vulnerabilities in IT systems that are unrelated to the path to that objective are lower priority, regardless of their CVSS scores.

Confidence in the finding: Was this finding confirmed by the scanner through active exploitation testing, or is it a version-based match? A scanner that checked the version number and matched it against a vulnerable range is less reliable than one that sent a specific exploit probe and confirmed the vulnerable behavior. High-confidence findings get prioritized over uncertain ones.

Cross-Referencing Multiple Scan Results

A professional scan workflow uses multiple tools, and their results often overlap — the same vulnerability reported by both OpenVAS and Nmap's NSE scripts increases confidence. More importantly, different tools sometimes find different things: OpenVAS might miss a web application vulnerability that Nuclei catches, while OpenVAS might identify OS-level patch gaps that Nuclei does not check.

The vulnerability analysis phase involves aggregating all scan results, deduplicating overlapping findings, cross-referencing with exploit databases, and producing a prioritized list of targets for the exploitation phase. This is analytical work that requires judgment and experience — it cannot be fully automated.

Continuous vs. Point-in-Time Scanning

Traditional vulnerability scanning is point-in-time: you scan at a specific moment, receive results for that moment, and those results are valid only as long as the environment does not change. A new server deployed the day after the scan is invisible to those results. A patch applied to a previously vulnerable system is not reflected until the next scan.

Modern enterprise vulnerability management programs move toward continuous scanning — using a combination of agents (which report vulnerability state in real time as software changes) and frequent scheduled scans to maintain an up-to-date picture of the vulnerability landscape.

In a penetration testing context, the scan is inherently point-in-time and reflects the state of the environment during the testing window. This is acknowledged in the penetration test report's scope and methodology section, and the report typically recommends implementing continuous vulnerability scanning as a remediation action.

3.3.9 Interpreting and Acting on Scan Results

Reading a Vulnerability Scan Report Like a Professional

The output of a vulnerability scan is not the end product — it is raw material that requires analysis and interpretation before it becomes actionable intelligence.

The anatomy of a vulnerability finding:

Every finding in a vulnerability scan report contains several elements. The CVE identifier ties the finding to the global vulnerability record. The CVSS score communicates severity. The plugin/NVT name describes the check that found it. The affected asset and service identify exactly what is vulnerable. The description explains what the vulnerability is and how it works. The evidence section shows what the scanner observed that led to the finding — this might be a version number, a service banner, or the actual response to a vulnerability probe. The solution section provides remediation guidance.

For each significant finding, a skilled penetration tester reads all of these elements and asks: Does this make sense? Is the evidence convincing? Could there be a reason this is a false positive? What are the actual exploitation requirements? What business impact would successful exploitation have?

Prioritizing Findings for Reporting

Not all vulnerabilities are equally urgent, and a penetration test report that lists 847 findings in CVSS score order without any analytical prioritization is not professionally valuable. The client's security team cannot act on 847 items simultaneously. They need guidance on where to focus first.

Professional prioritization considers CVSS score as a starting input, then adjusts based on exploitability (confirmed by manual testing or exploit availability), asset criticality (a vulnerability in the payment processing server is more critical than the same vulnerability in a non-production test server), exposure (internet-facing vs. internal), and business context (a confidentiality impact vulnerability is more critical for a data-heavy company than an availability impact).

The final report should present findings in priority tiers, with clear executive-level language explaining why the top-priority findings demand immediate attention.

Verification Before Reporting

The professional standard is to manually verify every finding before including it in a report. Verification does not always mean active exploitation — sometimes it means checking package manager logs to confirm a patch was not backported, reviewing service configurations to confirm the vulnerable condition exists, or checking vendor advisories to confirm the reported version range is accurate.

A finding that appears in scan output but cannot be manually verified should be reported as "potential vulnerability requiring further investigation" rather than a confirmed finding. This intellectual honesty protects both the penetration tester's credibility and the client's ability to triage effectively.

The Remediation Feedback Loop

Vulnerability scanning is not a one-time event — it is part of a continuous improvement cycle. After findings are remediated, the scanner should be run again to confirm that patches and configuration changes were effective. This rescan is the evidence that remediation worked.

In enterprise vulnerability management programs, this cycle runs continuously: scan, prioritize, remediate, rescan, verify, and repeat. The goal is a continuously shrinking attack surface as vulnerabilities are identified and fixed faster than new ones are introduced.

For a penetration tester, this manifests as the retest engagement: after the client remediates the critical and high findings from the initial assessment, the penetration testing firm returns to verify that the fixes were implemented correctly and effectively. A finding that was "patched" but still exploitable because the patch was applied incorrectly is a significant finding in the retest report.

Module 3 — Section 3.4: Understanding How to Analyze Vulnerability Scan Results

CompTIA PenTest+ / Ethical Hacking Certification Series
Professional Reference Guide — GitHub Edition
Module 3 Final Sections — The Complete Intelligence-to-Action Pipeline

3.4.1 Overview — From Raw Results to Actionable Intelligence
3.4.2 Sources for Further Investigation of Vulnerabilities
3.4.3 Investigating Vulnerability Information — Professional Workflow
3.4.4 How to Deal with a Vulnerability — The Full Decision Framework
3.5 Module 3 Summary — What You Have Learned and Why It Matters

3.4.1 Overview — From Raw Results to Actionable Intelligence

The Moment After the Scan Finishes

You have run your vulnerability scans. The tools have done their work. OpenVAS has generated a report with 312 findings. Nmap's NSE scripts flagged several critical vulnerabilities. Nuclei confirmed a handful of exposures. Nikto found default files on a web server. Searchsploit returned results for half a dozen service versions you detected.

Now what?

This is the moment that separates security practitioners who can run tools from those who can actually do security work. A list of 312 scanner findings is not intelligence. It is data — raw, unfiltered, partially inaccurate, and without context. Transforming that data into intelligence that can actually guide decisions — deciding what to fix, in what order, with what urgency, and with what evidence — requires understanding each vulnerability more deeply than any scanner can do automatically.

This is what Section 3.4 is about: the analytical work that happens after the scan.

The Core Problem with Scanner Output

Consider a real scenario. Your scan of a web application server returns forty findings. Among them:

A CVSS 9.8 finding for Apache Struts 2.5.26, flagged as vulnerable to CVE-2021-31805 — a remote code execution vulnerability. You look at the affected server. It turns out this is a Windows IIS server, not an Apache Struts application. The scanner misidentified a bundled library version in one response header. The finding is a false positive.

A CVSS 4.3 finding for an information disclosure issue — the server is returning the PHP version in every response header. This sounds low-severity from the score alone. But you check the PHP version: 5.6.40. PHP 5.6 reached end-of-life in December 2018. That version has received no security patches in over five years. There are dozens of unpatched critical vulnerabilities in PHP 5.6, but they are not flagged individually by the scanner because the scanner checked the PHP version, not every individual unpatched CVE in that version. The CVSS 4.3 finding is a signpost pointing to something far more serious.

A CVSS 7.5 finding for OpenSSH 7.4 — username enumeration via CVE-2018-15919. The server is an internal jump box accessible only from specific IP ranges on the management VLAN. From the external network, it is completely unreachable. The scanner does not know this topology detail. The finding is technically accurate but contextually lower priority than its CVSS score suggests, because exploitation requires network access the external attacker does not have.

These three examples illustrate the core problem: scanner output without contextual analysis misleads more than it guides. The numbers and colors in a scanner report are a starting point, not a conclusion.

What Professional Analysis Looks Like

When a professional penetration tester or vulnerability analyst receives scanner output, they apply a systematic analysis framework that addresses several questions for each significant finding:

Is this finding real, or is it a false positive? What evidence did the scanner produce, and does that evidence actually confirm the vulnerability?

Is this vulnerability actually exploitable in this specific environment? What conditions does exploitation require, and do those conditions exist here?

What does this vulnerability actually allow an attacker to do? The CVSS score describes a theoretical worst case — what is the realistic impact in this specific environment?

How does this finding relate to other findings? Does a chain of lower-severity vulnerabilities create a higher-impact attack path than any individual finding suggests?

What is the business context of the affected asset? A vulnerability on the payment processing server demands different urgency than the same vulnerability on a test server with no access to sensitive data.

What are the remediation options? Is a patch available? Can the service be reconfigured to eliminate the vulnerability? Are compensating controls available if immediate patching is not possible?

These questions are what Section 3.4 teaches you to answer. They require deep familiarity with vulnerability intelligence sources, an understanding of how vulnerabilities work, and the analytical discipline to treat scanner output as a hypothesis rather than a conclusion.

3.4.2 Sources for Further Investigation of Vulnerabilities

The Professional Reference Ecosystem

When a scanner reports a vulnerability, the professional next step is to look it up in authoritative sources to understand it fully before making any decisions about it. These sources are not optional reference material — they are the professional infrastructure of vulnerability management and penetration testing. Knowing where to find information and how to use each source is a core professional skill.

NVD — National Vulnerability Database

URL: https://nvd.nist.gov

Maintained by: National Institute of Standards and Technology (NIST), U.S. Department of Commerce

Purpose: The authoritative enrichment layer for CVE records. The primary reference for CVSS scores, CWE classifications, affected software configurations, and vendor advisory links.

The NVD is where you go first when you have a CVE identifier and need to understand it fully. Every CVE record in the NVD contains the CVSS v3.1 base score and vector string (giving you the full breakdown of attack vector, complexity, privileges required, user interaction, scope, and impact), the CWE identifier (classifying the type of underlying weakness), the CPE (Common Platform Enumeration) list of affected products and version ranges, links to vendor advisories and public references, and increasingly, EPSS scores reflecting exploitation likelihood.

The NVD record for CVE-2021-44228 (Log4Shell) shows CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H — the maximum possible score of 10.0. Reading this vector string tells you everything: it is exploitable remotely from anywhere on the internet, requires no special conditions to trigger, requires no account or authentication, requires no victim action, changes the security scope from the application to the operating system, and results in complete loss of confidentiality, integrity, and availability. That single vector string communicates the entire severity picture in standardized, unambiguous language.

One limitation to understand: NVD has faced processing backlogs at various points — in 2024 specifically, NIST fell significantly behind on enriching new CVE records with CVSS scores and CPE information. This created a period where many CVE records existed in the NVD without CVSS scores, forcing security teams to use alternative sources like VulnCheck or CISA's ADP (Authorized Data Publisher) enrichment. Knowing that NVD is the standard but not always the most current source is important professional awareness.

How to use NVD in practice:

# NVD API for automated lookups (useful in scripts)
curl "https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2021-44228"

# Search for vulnerabilities by keyword
curl "https://services.nvd.nist.gov/rest/json/cves/2.0?keywordSearch=apache+log4j"

# Get all critical CVEs published in the last 30 days
curl "https://services.nvd.nist.gov/rest/json/cves/2.0?cvssV3Severity=CRITICAL&pubStartDate=2024-01-01T00:00:00.000&pubEndDate=2024-01-31T23:59:59.000"

When reading an NVD record manually, always check the References section — it contains links to the vendor's official security advisory, patches, workarounds, and any proof-of-concept (PoC) code that has been publicly disclosed. These references are your roadmap for everything else you need to understand and act on the vulnerability.

MITRE CVE Database

URL: https://cve.mitre.org

Maintained by: MITRE Corporation, under sponsorship from CISA (Cybersecurity and Infrastructure Security Agency)

Purpose: The authoritative source for CVE identifiers — the source of record for CVE assignment and the canonical definition of each vulnerability.

While NVD enriches CVE records with scores and affected product data, MITRE's CVE Program is where CVE identifiers originate and where the definitive description of each vulnerability lives. The distinction matters: NVD can have processing delays, but MITRE's CVE list is updated more immediately when new CVEs are assigned.

MITRE also maintains CVE Numbering Authorities (CNAs) — organizations that have been authorized to assign CVE numbers for vulnerabilities in their own products. Major technology companies including Microsoft, Google, Apple, Cisco, Red Hat, and hundreds of others are CNAs. When Microsoft discovers a vulnerability in Windows, they assign it a CVE number themselves. When a researcher discovers a vulnerability in a product from a company that is not a CNA, they report it to MITRE or to a CNA with coordination responsibilities, who assigns the number.

Understanding the CNA ecosystem explains why you sometimes see CVEs for products you would not expect to have a formal disclosure process, and why the time between vulnerability discovery and CVE assignment varies dramatically.

CWE — Common Weakness Enumeration

URL: https://cwe.mitre.org

Maintained by: MITRE Corporation

Purpose: A categorization system for the underlying code-level weakness types that cause vulnerabilities.

If CVE is the dictionary of specific vulnerabilities ("CVE-2021-44228 is a specific Log4j flaw"), CWE is the grammar — the classification of the types of flaws that exist. CVE-2021-44228 is classified as CWE-917 (Improper Neutralization of Special Elements used in an Expression Language Statement). This classification tells you the root cause is that the application uses user-controlled input in a context where that input is interpreted as code.

CWE classifications are valuable for:

Root cause analysis: When multiple vulnerabilities share the same CWE, it indicates a systemic weakness in the codebase — the developers are repeatedly making the same class of mistake. This is valuable intelligence for recommending not just patches but improvements to development practices.

Defense strategy: Knowing the CWE category points directly to the defensive control that addresses it. CWE-89 (SQL Injection) points to parameterized queries. CWE-79 (Cross-Site Scripting) points to output encoding and Content Security Policy. CWE-78 (OS Command Injection) points to avoiding shell command construction with user input.

Tool coverage assessment: Static analysis tools (SAST) and security testing tools are often evaluated by which CWE categories they cover. Understanding CWE helps you evaluate whether your security tool suite has meaningful coverage.

Some critical CWEs every penetration tester must know:

CWE ID	Name	Common Manifestation
CWE-79	Cross-Site Scripting (XSS)	User input reflected in HTML without encoding
CWE-89	SQL Injection	User input concatenated into SQL queries
CWE-78	OS Command Injection	User input passed to shell execution functions
CWE-22	Path Traversal	User-controlled file paths allowing `../` traversal
CWE-287	Improper Authentication	Broken authentication mechanisms
CWE-306	Missing Authentication for Critical Function	Admin functions with no auth check
CWE-434	Unrestricted Upload of Dangerous File Type	File upload without type validation
CWE-502	Deserialization of Untrusted Data	Java/PHP deserialization vulnerabilities
CWE-611	XML External Entity (XXE)	XML parsers processing external entity declarations
CWE-798	Use of Hard-coded Credentials	Passwords embedded in source code

Exploit-DB

URL: https://www.exploit-db.com

Maintained by: Offensive Security (creators of Kali Linux and OSCP)

Purpose: A public archive of exploits and vulnerable software — the most comprehensive public database of working exploit code.

Exploit-DB is where you go when you need to know whether a working public exploit exists for a vulnerability and what it looks like. Finding that a vulnerability has a public exploit in Exploit-DB immediately elevates its priority — exploitation is no longer theoretical, and the code to do it is publicly available to anyone.

Each entry in Exploit-DB contains the vulnerability details, the affected software and version, the type of exploit (remote, local, web application, denial of service), a verification status (unverified or verified), the platform (Windows, Linux, multiple), and the actual exploit code or proof-of-concept.

The searchsploit command-line tool provides offline access to the Exploit-DB archive on Kali Linux, making it the fastest way to check exploit availability during a penetration test without requiring internet access.

# Search by software name
searchsploit apache tomcat
searchsploit openssh
searchsploit wordpress 5.8

# Search by CVE
searchsploit CVE-2021-44228
searchsploit CVE-2017-0144

# View exploit details without opening it
searchsploit -x 47837

# Copy exploit to current directory for use
searchsploit -m 47837

# Update the local database
sudo searchsploit --update

# Output as JSON for programmatic use
searchsploit --json apache tomcat 9.0 | python3 -m json.tool

# Search with full path output
searchsploit --path apache struts 2

Understanding Exploit-DB entry quality:

Exploit-DB entries vary in quality and reliability. Some exploits are mature, well-tested, and work reliably against multiple versions of the target software. Others are proof-of-concept code written by researchers to demonstrate a vulnerability exists — they may require significant modification to work in a real environment. Some may be incomplete, contain errors, or have been written for a slightly different version of the target than the one you are testing.

The "Verified" badge in Exploit-DB indicates the Offensive Security team has tested and confirmed the exploit works as described. Unverified exploits require more careful evaluation before trusting them in a professional engagement.

Metasploit Framework Database

URL: https://www.metasploit.com (framework); https://www.rapid7.com/db (vulnerability database)

Maintained by: Rapid7

Purpose: The most important single source for penetration-testing-ready exploit modules.

While Exploit-DB contains raw exploit code that often requires technical adaptation, the Metasploit Framework contains modules that have been engineered to work reliably across multiple target configurations, with built-in payloads, target selection, and auxiliary support. When a vulnerability has a Metasploit module, exploitation becomes significantly more accessible.

The Rapid7 vulnerability database at https://www.rapid7.com/db is the searchable online interface to the same data. You can search by CVE, by software, or by vulnerability type and see exactly which Metasploit modules exist, what platforms they target, how they are used, and what their reliability rating is.

For a penetration tester, the existence of a Metasploit module for a vulnerability is a critical piece of information for two reasons. First, it indicates the vulnerability is realistic and the exploitation path is well-understood. Second, it means any attacker with basic Metasploit skills can exploit it — raising the urgency for remediation.

# Within Metasploit console:
msfconsole

# Search for modules by CVE
search CVE-2021-44228
search CVE-2017-0144

# Search by name or description
search eternalblue
search log4shell
search struts

# Get detailed information about a module
info exploit/multi/handler
info exploit/windows/smb/ms17_010_eternalblue

# Check module reliability ratings
show all               # All modules
use exploit/windows/smb/ms17_010_eternalblue
info                   # Shows rank: Excellent/Great/Good/Normal/Average/Low/Manual

Metasploit reliability rankings: Modules are rated Excellent, Great, Good, Normal, Average, Low, or Manual. "Excellent" means the exploit never crashes services — it is safe to use. "Normal" means the exploit is reliable but may crash services if it fails. "Manual" means the exploit is for educational purposes and requires significant operator skill and manual steps. For production penetration tests, understanding these ratings helps you assess operational risk.

CISA KEV — Known Exploited Vulnerabilities Catalog

URL: https://www.cisa.gov/known-exploited-vulnerabilities-catalog

Maintained by: Cybersecurity and Infrastructure Security Agency (CISA), U.S. Federal Government

Purpose: A catalog of CVEs that have been confirmed as actively exploited in the wild — the most authoritative real-world exploitation signal available.

The CISA KEV catalog was established in November 2021 and has become one of the most important vulnerability prioritization signals in the industry. Unlike CVSS scores (which represent theoretical severity) or EPSS scores (which represent statistical exploitation probability), the KEV catalog lists vulnerabilities that are known, with confirmed evidence, to have been actively exploited by threat actors.

U.S. federal civilian agencies are mandated by CISA Binding Operational Directive 22-01 to remediate KEV-listed vulnerabilities within specified timeframes (typically 2 weeks for new additions). But the KEV's value extends far beyond federal compliance — it is the clearest available signal that a vulnerability is being weaponized in real attacks.

How KEV entries are added: CISA adds a vulnerability to the KEV catalog when there is reliable evidence of active exploitation. This evidence comes from multiple sources: CISA's own threat intelligence, reports from federal agencies, commercial threat intelligence providers, information sharing partnerships, and public reporting from security researchers. The bar for inclusion is confirmed exploitation — not theoretical exploitability.

Why KEV matters for penetration testing:

When you find a KEV-listed vulnerability in a client's environment, the finding has an additional dimension that elevates it beyond the CVSS score: real threat actors are actively using this exact vulnerability in real attacks right now. The remediation urgency is not a theoretical calculation — it is based on confirmed attacker behavior.

# Fetch the KEV catalog as JSON
curl -s "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json" | python3 -m json.tool

# Check if a specific CVE is in KEV
curl -s "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json" | \
  python3 -c "
import json, sys
data = json.load(sys.stdin)
target_cve = 'CVE-2021-44228'
for v in data['vulnerabilities']:
    if v['cveID'] == target_cve:
        print(f'FOUND IN KEV: {target_cve}')
        print(f'Product: {v[\"product\"]}')
        print(f'Vendor: {v[\"vendorProject\"]}')
        print(f'Date Added: {v[\"dateAdded\"]}')
        print(f'Due Date: {v[\"dueDate\"]}')
        print(f'Notes: {v[\"notes\"]}')
"

EPSS — Exploit Prediction Scoring System

URL: https://www.first.org/epss

Maintained by: FIRST (Forum of Incident Response and Security Teams) in partnership with threat intelligence contributors

Purpose: A machine learning-based system that predicts the probability that a CVE will be exploited in the next 30 days.

EPSS was developed to solve a specific problem: CVSS scores measure theoretical severity but have weak correlation with actual exploitation. A CVE with a CVSS score of 9.8 might have a very low probability of being exploited in practice — perhaps because the affected software is rarely deployed, or because exploitation requires conditions that are almost never present, or because no public exploit exists. Meanwhile, a CVE with a CVSS score of 6.5 might have a very high EPSS score — because it has a mature Metasploit module, affects widely-deployed software, and is already being actively used in attacks.

The EPSS model analyzes hundreds of signals to generate its daily score: the CVE's CVSS metrics, the type of weakness (CWE), whether the CVE is listed in threat intelligence feeds, whether public exploit code exists, whether it has Metasploit modules, whether it is discussed in offensive security tooling, and historical exploitation patterns for similar vulnerability types.

EPSS produces two values:

The score (0 to 1) represents the probability of exploitation activity in the next 30 days. A score of 0.97 means there is a 97% probability that this vulnerability will be observed in exploitation attempts in the next 30 days.

The percentile represents where this vulnerability ranks within the universe of all CVEs. A percentile of 0.99 means this vulnerability has a higher EPSS score than 99% of all CVEs. This context matters: even a seemingly low EPSS score of 0.05 might be in the 85th percentile, meaning it has a higher exploitation probability than 85% of all vulnerabilities.

The practical insight EPSS provides:

Research has shown that only about 2-5% of all published CVEs are actually exploited in the wild in any given period. Prioritizing remediation based purely on CVSS score means you are devoting resources to patching vulnerabilities that will likely never be exploited, while potentially missing lower-CVSS vulnerabilities that are actively being weaponized. EPSS helps distinguish between "this is theoretically severe" and "this is being actively attacked."

# Fetch EPSS score for a specific CVE
curl "https://api.first.org/data/v1/epss?cve=CVE-2021-44228"

# Fetch EPSS scores for multiple CVEs
curl "https://api.first.org/data/v1/epss?cve=CVE-2021-44228,CVE-2017-0144,CVE-2019-0708"

# Get CVEs with highest EPSS scores (top 100)
curl "https://api.first.org/data/v1/epss?order=!epss&limit=100"

# Get all CVEs with EPSS score above 0.9 (very high exploitation likelihood)
curl "https://api.first.org/data/v1/epss?epss-gt=0.9&limit=500"

Vendor Security Advisories

Every major software vendor publishes their own security advisories — formal documents describing vulnerabilities in their products, their severity, and the available remediation. These are some of the most authoritative and detailed vulnerability information sources available, because the vendor wrote the vulnerable code and knows it better than anyone.

Microsoft Security Response Center (MSRC): https://msrc.microsoft.com/update-guide

Published monthly on Patch Tuesday (second Tuesday of each month). Contains every vulnerability addressed in that month's updates, with CVSS scores, affected versions, and remediation guidance. The CVE details here are often more detailed than NVD — Microsoft frequently includes FAQs about specific exploitation scenarios, whether exploitation has been observed in the wild, and whether the vulnerability is publicly known.

Cisco Security Advisories: https://sec.cloudapps.cisco.com/security/center/publicationListing.x

Cisco publishes advisories in a tiered severity system (Critical, High, Medium, Low) and includes details specific to their product ecosystem. For any Cisco network device vulnerability found during a scan, the Cisco advisory is the authoritative source for the correct workaround or upgrade path.

Red Hat Security Advisories: https://access.redhat.com/security/security-updates

Red Hat's advisories are particularly valuable for understanding the impact of backported patches — Red Hat explicitly documents which backported fixes are included in each RHSA update, clarifying exactly which CVEs are addressed even when the version number has not changed.

Apache Security Advisories: https://httpd.apache.org/security/vulnerabilities_24.html

Direct from the Apache HTTP Server project — essential when investigating web server vulnerabilities.

Ubuntu Security Notices (USN): https://ubuntu.com/security/notices

Debian Security Advisories (DSA): https://www.debian.org/security

Both provide package-level patch information that explains exactly which CVEs are addressed in each security update.

The habit of consulting vendor advisories:

When a scanner reports a vulnerability in vendor software, the sequence should be: NVD for the authoritative CVSS analysis, then the vendor advisory for the specific remediation path. The vendor advisory tells you the exact version to upgrade to, whether a workaround exists, and whether the vulnerability has been exploited in the wild (vendors sometimes report this in their advisories before it appears elsewhere).

PoC-in-GitHub and Research Resources

The security research community publishes proof-of-concept code on GitHub almost immediately after major vulnerability disclosures. This code varies enormously in quality and intent — some is academic research, some is functional exploit code intended for authorized testing, and some is weaponized malware. A penetration tester needs to understand this ecosystem.

GitHub vulnerability monitoring:

# Search GitHub for PoC code for a specific CVE
# This can be done via the GitHub API
curl "https://api.github.com/search/repositories?q=CVE-2021-44228&sort=stars"

# GitHub has also introduced a Security Advisories database
# accessible via: https://github.com/advisories
# This is particularly valuable for open-source ecosystem vulnerabilities

Security research blogs worth following:

Google Project Zero (https://googleprojectzero.blogspot.com) publishes deep technical analysis of zero-day vulnerabilities discovered by Google's elite research team. Their write-ups are technically demanding but represent the state of the art in vulnerability research.

Qualys ThreatLabs (https://blog.qualys.com/vulnerabilities-threat-research) publishes rapid analysis of newly disclosed vulnerabilities, often within hours of public disclosure.

Tenable Research (https://www.tenable.com/blog) publishes both vulnerability analysis and scanner plugin development context — useful for understanding exactly what the scanner is checking.

MITRE ATT&CK Framework

URL: https://attack.mitre.org

Maintained by: MITRE Corporation

Purpose: A knowledge base of adversary tactics, techniques, and procedures (TTPs) derived from real-world observations of attacker behavior.

ATT&CK is different from the CVE/CWE ecosystem in a fundamental way: while CVE describes specific vulnerabilities and CWE describes weakness classes, ATT&CK describes what attackers do with vulnerabilities — the actions, movements, and techniques observed in real intrusions.

ATT&CK organizes attacker behavior into 14 Tactics (the high-level goals like Initial Access, Execution, Persistence, Privilege Escalation, Defense Evasion, Credential Access, Discovery, Lateral Movement, Collection, Command and Control, Exfiltration, and Impact). Each tactic contains multiple Techniques describing specific methods attackers use to achieve that goal.

For vulnerability analysis, ATT&CK provides context. When you find a vulnerability, ATT&CK helps you understand which ATT&CK techniques it enables. EternalBlue (CVE-2017-0144) maps to T1210 (Exploitation of Remote Services) under Lateral Movement — understanding this tells you that finding EternalBlue not just on externally exposed systems but on internal Windows hosts is critical, because it enables attacker lateral movement across the internal network.

MITRE D3FEND: The defensive counterpart to ATT&CK, MITRE D3FEND (https://d3fend.mitre.org) maps defensive countermeasures to ATT&CK techniques. For each ATT&CK technique an attacker might use to exploit a vulnerability, D3FEND identifies what defensive controls (network segmentation, authentication hardening, endpoint monitoring) would detect or prevent it.

Threat Intelligence Platforms — Commercial and Open Source

Beyond the standard databases, commercial and open-source threat intelligence platforms provide enriched, real-time vulnerability intelligence that integrates multiple data sources.

VirusTotal Intelligence: Beyond its malware scanning function, VirusTotal maintains a large dataset of vulnerability-related intelligence — which malware families exploit which CVEs, which threat actor groups are associated with which vulnerabilities, and how widely a vulnerability is being exploited.

Shodan CVE Search: Shodan maintains data on vulnerable internet-exposed systems and can show you how many internet-facing hosts are running versions vulnerable to specific CVEs. This helps contextualize findings: finding Log4Shell on an internal server is serious, but knowing that 100,000 internet-facing servers are also vulnerable helps communicate the broader threat landscape.

GreyNoise: Aggregates and analyzes internet-wide scanner activity to distinguish between mass exploitation (attackers scanning everything) and targeted attacks. When you see a GreyNoise tag on a CVE indicating mass scanning, it means opportunistic attackers are already actively probing for this vulnerability at internet scale.

Recorded Future, Mandiant Advantage, CrowdStrike Falcon Intelligence: Commercial threat intelligence platforms that provide attribution, campaign tracking, and deeper analysis of which threat actor groups are using which vulnerabilities. Used in enterprise security operations and red team engagements that require threat-actor-specific context.

3.4.3 Investigating Vulnerability Information — Professional Workflow

The Vulnerability Investigation Process

When a scanner reports a finding, the professional investigation process follows a consistent sequence that ensures every significant finding is fully understood before any action is taken.

Step 1: Extract the precise version information

The scanner's finding includes the CVE identifier and usually the version that triggered the finding. Start by confirming: is the version information accurate? What is the exact version of the software running, and how was it determined?

For a web server, confirm the version via multiple methods: the HTTP Server header, the page source (some CMSes disclose versions in meta tags), version-specific behavior fingerprinting (some features only appear in specific versions), and if you have system access, the package manager:

# Linux: Check installed package version
dpkg -l apache2           # Debian/Ubuntu
rpm -qa | grep httpd      # Red Hat/CentOS
apt-cache policy apache2  # Ubuntu: shows installed and candidate versions

# Check for backported patches specifically
apt-cache show apache2 | grep -i changelog
# Or look at the full changelog
zcat /usr/share/doc/apache2/changelog.Debian.gz | head -50

Step 2: Look up the CVE in NVD

Go to https://nvd.nist.gov/vuln/detail/[CVE-ID] and read the complete record. Note the CVSS vector string (not just the score), the CWE classification, and all the reference links. If the CVSS vector says PR:H (Privileges Required: High), the vulnerability requires administrator access to exploit — this is a critical detail that changes the risk calculation.

Step 3: Check the vendor advisory

Find the official vendor security advisory for this CVE. The vendor advisory is the authoritative statement on: which exact versions are affected, whether the vulnerability has been exploited in the wild, what the remediation is (specific version to upgrade to), and whether any workarounds exist.

Step 4: Check CISA KEV

Is this CVE in the CISA Known Exploited Vulnerabilities catalog? If yes, there is confirmed evidence of active exploitation — this finding gets elevated priority regardless of CVSS score.

Step 5: Check EPSS

What is the EPSS score? A high EPSS score (particularly above 0.5) combined with a KEV listing is the strongest possible signal for immediate remediation. A low EPSS score (below 0.1) on a high-CVSS vulnerability suggests it remains theoretical — still worth addressing but with less urgency.

Step 6: Check Exploit-DB and Metasploit

# Check for public exploits
searchsploit CVE-2021-44228
searchsploit -j CVE-2021-44228  # JSON output

# Check Metasploit
msfconsole -q -x "search CVE-2021-44228; exit"

If a Metasploit module exists with an "Excellent" or "Great" reliability rating, exploitation is accessible to anyone with basic penetration testing skills.

Step 7: Verify the finding manually if possible

Depending on the engagement scope and the type of finding, attempt to verify the vulnerability manually. For a Heartbleed finding, send the crafted TLS heartbeat probe and observe the response. For a Log4Shell finding, send a JNDI lookup probe and monitor your callback server for a DNS request. For an EternalBlue finding, use the Metasploit smb-vuln-ms17-010 scanner auxiliary module (not the exploit).

Manual verification converts a potential finding into a confirmed finding, which is a fundamental difference in a penetration test report.

Step 8: Document everything

Every step of this investigation should be documented: what you found, what sources you consulted, what evidence you gathered, what you concluded, and whether the finding is confirmed or suspected. This documentation is the foundation of your report.

A Practical Investigation Example

Suppose your OpenVAS scan reports this finding on host 10.10.10.50:

"OpenSSH 7.4 – Username Enumeration (CVE-2018-15919)" — CVSS 5.3 — Medium

Here is how the investigation proceeds:

First, you look up CVE-2018-15919 in NVD. The vector string is CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N — network exploitable, low complexity, no privileges or user interaction required, but only low confidentiality impact with no integrity or availability impact. Score: 5.3. The CWE is CWE-203 (Observable Discrepancy) — the server responds differently to valid vs. invalid usernames during authentication attempts.

Second, you check whether this host is publicly accessible. It is SSH on an internal management server, reachable only from the management VLAN. An external attacker cannot reach it.

Third, you check CISA KEV. CVE-2018-15919 is not in the catalog.

Fourth, you check EPSS. Score: 0.002 (0.2% exploitation probability). This is low.

Fifth, you check Exploit-DB. There is a public PoC demonstrating username enumeration. There is no Metasploit module.

Your analysis conclusion: This is a real vulnerability. However, the network location (internal management VLAN only) limits the attack surface dramatically. An attacker would need to be on the management VLAN to exploit it, and by that point they likely have more impactful attack paths available. It is a valid Medium finding — it enables username enumeration for credential attacks against SSH — but it is not a priority remediation target compared to the Critical findings elsewhere in the assessment.

This analysis takes perhaps ten minutes. The conclusion is meaningfully different from "CVSS 5.3 → Medium → patch next quarter" because it incorporates context the scanner cannot know.

3.4.4 How to Deal with a Vulnerability — The Full Decision Framework

The Four Responses to a Confirmed Vulnerability

When a vulnerability has been confirmed and fully analyzed, there are exactly four ways an organization can respond. Every vulnerability that is not addressed by patching falls into one of the other three categories — and all four require explicit documentation and owner accountability.

Understanding these four options is fundamental to professional vulnerability management and penetration testing reporting. Your recommendations to clients must acknowledge that patching is not always immediately possible and guide them through the complete decision framework.

Response 1: Remediation (Patching and Configuration Fixes)

Remediation means eliminating the vulnerability. The primary form is patching — applying the vendor-released software update that fixes the vulnerability. But remediation also includes configuration changes that remove the vulnerable condition without requiring a software update.

Software patching:

Patching sounds simple in principle. In practice, it involves multiple steps that cannot be skipped without risking production disruption:

Testing before production deployment is non-negotiable for any system with business-critical dependencies. A patch that breaks application compatibility creates its own incident. The test environment should match production as closely as possible — same OS version, same application version, same dependent libraries. Apply the patch to the test environment first, run the application's full test suite, confirm nothing breaks, and only then proceed to production.

Patch deployment process in a managed enterprise environment goes through a formal change management workflow: change request, technical review, approval, scheduled maintenance window, deployment, rollback plan, and post-deployment verification. A penetration test report that recommends patching without acknowledging this process is not providing actionable guidance — it is providing an aspiration.

Emergency patching for Critical/KEV vulnerabilities compresses this timeline. An organization's security policy should define an expedited patching process for Critical vulnerabilities — perhaps a 48-72 hour process that skips some normal change management bureaucracy while maintaining essential safeguards. The default patch management SLA (which might be "30 days for critical") is not appropriate for a vulnerability being actively exploited in the wild.

Configuration-based remediation:

Some vulnerabilities can be addressed without patching, by reconfiguring the service to eliminate the vulnerable behavior:

Disabling a vulnerable feature (SSL 2.0, TLS 1.0, SNMP v1, anonymous LDAP binding, FTP, Telnet — none of these need to be running in a secure environment).

Restricting network access so the vulnerable service cannot be reached by unauthorized parties (firewall rules, network segmentation).

Removing an unnecessary service entirely — if a service is not needed, removing it is better than patching it.

Enabling authentication on a service that was running without it.

The configuration remediation advantage: These changes can often be implemented immediately without waiting for a vendor patch or going through the full patch testing cycle. When a critical vulnerability has no available patch (zero-day scenario) or when patching would break critical functionality, configuration changes may be the fastest available remediation path.

Response 2: Mitigation (Compensating Controls)

Mitigation does not eliminate the vulnerability but reduces the likelihood or impact of exploitation. Compensating controls are the mitigations implemented when immediate remediation is not possible.

Common compensating controls:

Network segmentation and firewall rules: If a vulnerable service cannot be immediately patched, restrict network access so only authorized systems can reach it. A vulnerable database server that can only be reached from application servers — not from the internet or from end-user workstations — has a dramatically reduced exposure even if the database software itself remains unpatched.

Web Application Firewalls (WAF): A WAF can detect and block exploit payloads before they reach a vulnerable application. When a zero-day web application vulnerability is disclosed and no patch is available, vendors often publish WAF rules as temporary protection while the patch is developed and tested. This is not a permanent solution — WAF rules can be bypassed — but it buys time.

Intrusion Detection and Prevention Systems (IDS/IPS): If exploitation of a vulnerability produces distinctive network signatures, an IDS/IPS with updated rules can detect and block exploitation attempts. The limitation is that sophisticated attackers modify their exploit code specifically to evade IDS signatures.

Disabling specific features without full patching: Some vulnerabilities only affect specific features or configurations. If the vulnerable feature is not needed, disabling it eliminates the attack vector without requiring the full software update.

Enhanced monitoring: When a vulnerability cannot be immediately patched and compensating controls are limited, increasing monitoring around the vulnerable asset provides earlier detection of exploitation. This does not prevent the attack but reduces the response time — in security, detection speed is a meaningful security metric.

The important limitation of compensating controls: They are temporary and partial. Every organization that implements a compensating control instead of patching must document:

The vulnerability being mitigated
The compensating control implemented
The residual risk that remains
The timeline for actual remediation
The person accountable for driving remediation to completion

Without this documentation and accountability, compensating controls become indefinite deferrals — and deferred vulnerabilities eventually become breach vectors.

Response 3: Risk Acceptance

Risk acceptance means explicitly deciding not to remediate a vulnerability and formally accepting the residual risk. This is a legitimate business decision in specific circumstances, but it must be explicit, documented, and authorized at an appropriate level of the organization.

When risk acceptance is appropriate:

End-of-life systems scheduled for retirement: If a system with a critical vulnerability is scheduled to be decommissioned in 30 days, the business case for emergency patching is weak. The remediation effort and risk of production disruption from patching may be greater than the risk of exploitation during the 30-day retirement period — particularly if compensating network controls restrict access.

Cost-prohibitive remediation: Occasionally the remediation pathway for a vulnerability is technically complex, has a high risk of breaking critical functionality, or requires significant architectural changes that cannot be implemented quickly. If the business cost of remediation exceeds the business risk of the vulnerability, risk acceptance may be appropriate — but this decision must be made by senior leadership with clear visibility into the risks they are accepting.

Very low EPSS and no realistic attack path: A vulnerability in an internal system with no realistic external attack path, a very low EPSS score, and no known exploitation in the wild may be appropriate for risk acceptance with compensating controls. This is not a justification for laziness — it is a proportionality calculation.

What risk acceptance is NOT:

Risk acceptance is not the same as ignoring a finding. The distinction is documentation and authorization:

An ignored finding: nobody documented it, nobody decided to accept the risk, nobody is monitoring it, nobody knows whether it was ever addressed.
A risk-accepted finding: documented in the risk register, formally approved by the CISO or a designated risk authority, compensating controls identified and implemented, monitoring established, and a review date set to reassess whether the accepted risk level remains appropriate.

In PCI DSS environments, risk acceptance for findings in the Cardholder Data Environment requires formal documentation and may require QSA review. In HIPAA environments, risk acceptance for ePHI-related vulnerabilities must be part of the formal risk analysis and risk management plan.

Response 4: Transfer (Risk Transfer)

Risk transfer moves the financial consequence of a vulnerability's exploitation to a third party — typically through cybersecurity insurance (cyber liability insurance). The vulnerability itself is not eliminated, but if exploitation results in a breach with financial consequences (incident response costs, notification costs, regulatory fines, business interruption), the insurance policy covers those costs up to the policy limit.

Risk transfer is a legitimate part of a comprehensive risk management program, but it has important limitations in the context of vulnerability management:

Cyber insurance underwriters increasingly require organizations to demonstrate baseline security hygiene before issuing policies. Unpatched Critical and High vulnerabilities — particularly KEV-listed ones — may result in policy denial or exclusion clauses. If you suffer a breach through a KEV-listed vulnerability that you knew about and did not patch, many cyber insurance policies will not pay out.

Risk transfer does not protect reputation. The financial cost of a breach may be covered by insurance, but the reputational damage, customer trust erosion, and regulatory scrutiny that follows a publicly disclosed breach cannot be transferred.

Risk transfer does not stop the attack. The breach still occurs, customer data is still compromised, operations are still disrupted — insurance covers the cleanup costs after the fact, not the harm caused during the incident.

In the context of a penetration test report, risk transfer is rarely a recommendation for specific technical vulnerabilities. It is a strategic-level recommendation for the overall security program — "we recommend ensuring your cyber liability insurance policy covers incidents arising from vulnerabilities in externally-accessible systems."

The Prioritization Matrix — Making Decisions Under Pressure

In a real organization, there are always more vulnerabilities than there are resources to remediate them. A mature vulnerability management program needs a principled way to decide which vulnerabilities get resources first when everything cannot be patched simultaneously.

The professional prioritization framework combines four dimensions:

Severity (from CVSS): How bad is the vulnerability in the theoretical worst case?

Exploitation likelihood (from EPSS + KEV + Exploit availability): How likely is this vulnerability to be actually exploited?

Asset criticality: How important is the affected system to business operations? A vulnerability in the payment processing system demands more urgent attention than the same vulnerability in an employee blog.

Exposure: Is the affected service publicly accessible from the internet? Or is it accessible only internally, or only from specific management networks?

These four dimensions create a prioritization matrix. The highest-priority vulnerabilities are those that score high on all four: Critical CVSS, high EPSS, on a business-critical system, and internet-facing. The lowest-priority are those that score low on all four: Low CVSS, very low EPSS, on a non-critical system, with no external exposure.

Priority Level	Criteria	Typical Response Time
Immediate	Critical/High CVSS + KEV listed + internet-facing	24–72 hours
Urgent	Critical CVSS + EPSS > 0.5 + business-critical asset	7 days
High	High CVSS + exploit available + production system	14 days
Standard	Medium–High CVSS + limited exposure	30 days
Planned	Low–Medium CVSS + internal only + low EPSS	Next maintenance cycle
Accept/Monitor	Low CVSS + very low EPSS + non-critical asset	Risk acceptance with monitoring

Communicating Vulnerability Risk to Non-Technical Stakeholders

One of the most practically important skills in professional cybersecurity — and one that is rarely taught directly — is translating technical vulnerability findings into business language that executive and management audiences can understand and act on.

A CISO or CEO does not need to understand what AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H means. They need to understand the business answer to three questions: What is at risk? What happens if we do not fix this? What does fixing it require?

Technical finding: CVE-2021-44228 (Log4Shell) detected on the public-facing customer portal. CVSS 10.0. CISA KEV listed. EPSS 0.97. Metasploit module available with Excellent reliability.

Executive translation: "We discovered a critical vulnerability in the software running our customer portal. This vulnerability requires no password or login — any person with internet access can use it to gain complete control of the server. It is actively being used in attacks against organizations worldwide right now. The fix is a software update that our team can deploy within 24 hours. The risk of not patching immediately is complete compromise of the customer portal server, potentially including customer data exposure and ransomware deployment."

This translation requires the technical professional to understand not just the vulnerability itself but the business context: what data is on the affected server, what operations depend on it, what regulatory consequences would follow a breach, and what the remediation process actually involves in operational terms.

Verification and Retesting — Closing the Loop

After remediation is implemented, the vulnerability lifecycle is not complete until the fix has been verified. Verification involves:

Rescanning with the same tools that originally detected the finding. If the scan now comes back clean, the automated check confirms the remediation was applied. This is necessary but not sufficient — scanner checks are not infallible.

Manual verification confirms the vulnerability can no longer be exploited using the same technique that would have been used to exploit it. For an EternalBlue finding, this means sending the specific SMB negotiation sequence that triggers the vulnerability and confirming the response indicates a patched system. For a Heartbleed finding, this means sending the malformed heartbeat and confirming the server does not return excess data.

Evidence documentation for compliance purposes records what the vulnerability was, when it was discovered, what remediation was implemented, when it was implemented, and how verification was performed. This audit trail is required for PCI DSS, HIPAA, SOC 2, and most other compliance frameworks.

In a penetration testing engagement context, this becomes the retest — a follow-up engagement specifically to verify that the highest-priority findings have been effectively remediated. A retest finding that confirms a critical vulnerability was patched correctly is valuable positive evidence. A retest finding that the critical vulnerability was supposedly patched but is still exploitable is one of the most important findings a penetration test can produce.

3.5 Module 3 Summary — What You Have Learned and Why It Matters

3.5.1 What You Learned in Module 3

Module 3 — Information Gathering and Vulnerability Scanning — built the complete intelligence-gathering and vulnerability discovery capability that forms the technical foundation of every penetration testing engagement and vulnerability assessment.

From Section 3.1 — Performing Passive Reconnaissance:

You learned that the reconnaissance phase can be divided into two fundamentally different approaches based on whether they interact with the target's systems. Passive reconnaissance collects information from public and third-party sources without generating any traffic to the target, making it legally safe before authorization is in place and undetectable by any security monitoring the target has deployed.

You learned the OSINT methodology — that professional intelligence gathering follows the intelligence cycle (planning, collection, processing, analysis, dissemination, and feedback) and that the pivot model transforms every discovered data point into additional intelligence. Starting with only an organization name, a skilled analyst can map the complete external attack surface without touching a single target system.

You learned to extract intelligence from DNS records at a depth that most practitioners miss. A single DNS zone can reveal the email provider, cloud infrastructure, subdomain inventory, administrator email addresses, third-party service integrations, email security posture (through DMARC policy), and CA authorization policies — all before the first probe reaches a target system.

You learned the OSINT tool ecosystem: OSINT Framework as the structured methodology map, recon-ng as the professional modular reconnaissance platform, SpiderFoot as the automated multi-source intelligence aggregator, Shodan as the Internet-connected device search engine, and the complete breach intelligence stack from HaveIBeenPwned through specialized tools like WhatBreach, PwnDB, and Buster.

You learned that SSL certificates are intelligence sources — that the Subject Alternative Names in a certificate can reveal an organization's entire subdomain inventory, and that Certificate Transparency logs provide historical certificate data revealing infrastructure evolution over years.

You learned social media intelligence gathering at a professional level — that LinkedIn job listings may be the single richest source of technology stack intelligence, that employee profiles reveal specific versions of technologies in use, and that this intelligence enables precisely targeted technical attacks and highly credible social engineering campaigns.

From Section 3.2 — Performing Active Reconnaissance:

You learned that active reconnaissance begins the moment packets are sent to a target system — that every active technique generates log entries, potentially triggers alerts, and requires explicit authorization before beginning.

You learned Nmap not as a tool but as a protocol manipulation framework. You understood each scan type from first principles: why a SYN scan is "half-open," why FIN/NULL/Xmas scans do not work against Windows, what the ACK scan actually measures (firewall rules, not port states), and why the UDP scan is simultaneously the most important and most technically challenging scan type.

You learned that timing options are not just about speed — they are a stealth/speed trade-off with direct implications for detection avoidance, and that -T0 (one probe every five minutes) completely evades time-based IDS correlation while making a full scan take days.

You learned enumeration as the discipline of extracting service-specific intelligence from discovered open ports: SNMP enumeration as a potential single-query network topology disclosure, SMB enumeration revealing user accounts and password policy, LDAP enumeration mapping Active Directory structure, and web service enumeration establishing the foundation for application-layer testing.

You learned Scapy as the framework for understanding network protocols at the byte level and constructing custom packets for situations where standard tools cannot provide the precise interaction needed. You understood that Scapy's power comes from removing the abstraction layers between the operator and the network protocol itself.

You learned network eavesdropping — that ARP poisoning enables traffic interception on switched networks by poisoning the Layer 2 address resolution tables of communicating hosts, that unencrypted protocols including HTTP, FTP, Telnet, and SNMPv1/v2 pass credentials in plaintext visible to any positioned observer, and that Wireshark's display filter language enables precise extraction of relevant packets from millions of captured frames.

From Section 3.3 — Understanding the Art of Performing Vulnerability Scans:

You learned the fundamental distinction between vulnerability scanning (automated, broad, reports known weaknesses) and penetration testing (manual, targeted, proves real-world exploitability and business impact). This distinction is not academic — it determines what clients should expect from each type of engagement and what questions each answers.

You learned how vulnerability scanners work mechanically: discovery and asset inventory, fingerprinting and version detection, vulnerability checks via plugins and NVTs, and scoring and reporting using CVSS. Understanding this architecture enables intelligent tool use — knowing why authenticated scans produce better results, why version-matching leads to false positives, and why keeping plugin databases current is not optional.

You learned the CVE and CVSS systems as a professional language. You can now read a CVSS vector string and understand exactly what it communicates: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H is the full severity picture of Log4Shell, encoding not just the score but the attack vector, complexity, authentication requirements, scope impact, and CIA consequences.

You learned the professional vulnerability scanning tool ecosystem: Nessus as the commercial standard, OpenVAS/GVM as the professional-grade free alternative, Nuclei as the modern template-based scanner excelling at web application and API vulnerability discovery, and Nikto as the fast web server configuration scanner.

You learned the seven critical challenges of vulnerability scanning: false positives (the scanner cried wolf), false negatives (the scanner missed the elephant), production system impact, scope and coverage gaps, database currency requirements, and the authenticated versus unauthenticated trade-off. These challenges explain why scanner output requires human analysis and why penetration testing remains irreplaceable even for organizations with mature vulnerability scanning programs.

From Section 3.4 — How to Analyze Vulnerability Scan Results:

You learned the professional intelligence source ecosystem: NVD for authoritative CVSS analysis, MITRE CVE as the source of record, CWE for root cause classification, Exploit-DB for public exploit availability, Metasploit for penetration-testing-ready modules, CISA KEV for confirmed real-world exploitation evidence, and EPSS for machine-learning-based exploitation probability prediction.

You learned that CVSS scores and EPSS scores answer different questions. CVSS asks "how bad is this vulnerability in the theoretical worst case?" EPSS asks "what is the probability this vulnerability will actually be exploited in the next 30 days?" Only about 2-5% of published CVEs are exploited in practice — prioritizing remediation based on CVSS alone means devoting resources to vulnerabilities that will likely never be exploited while potentially deprioritizing actively weaponized lower-scoring vulnerabilities.

You learned the complete vulnerability investigation workflow: version verification, NVD lookup, vendor advisory consultation, CISA KEV check, EPSS assessment, Exploit-DB and Metasploit check, manual verification, and documentation. This workflow transforms scanner output from raw data into confirmed intelligence.

You learned the four responses to a confirmed vulnerability — remediation, mitigation, risk acceptance, and risk transfer — and the professional standard for each. You learned that risk acceptance is not the same as ignoring a finding, that compensating controls require explicit documentation and accountability, and that the remediation verification loop (rescan + manual verification + documentation) is what closes the vulnerability lifecycle.

3.5.2 How These Skills Connect to the Rest of Your Career

The skills built in Module 3 are not just tools for passing a certification exam. They are the practical core of what security professionals do every day across multiple roles.

As a penetration tester, everything in Module 3 happens before you write a single exploit. Your success rate in the exploitation phase is a direct function of how thoroughly and accurately you completed the reconnaissance and scanning phases. The best penetration testers spend more time on recon than on exploitation — because thorough recon means precise, efficient, high-confidence exploitation rather than random probing.

As a security analyst in a SOC or vulnerability management team, you use these same intelligence sources daily. CISA KEV alerts drive emergency patching decisions. EPSS scores inform prioritization when the queue of scanner findings exceeds the remediation capacity. Vendor advisories determine the accurate remediation path. The analytical framework for distinguishing real findings from false positives is the foundation of an effective vulnerability management program.

As a red team operator, passive reconnaissance capabilities determine how much intelligence you can gather without detection. The ability to identify technology stacks, subdomains, email infrastructure, and employee details from entirely public sources, before making any contact with the target, is one of the most valuable capabilities in advanced threat simulation.

As a security engineer or architect, understanding how vulnerabilities are discovered, categorized, and exploited informs defensive architecture decisions. Why network segmentation limits lateral movement. Why patching timelines need to be aligned with EPSS scores and KEV listings rather than quarterly maintenance cycles. Why authenticated scanning is worth the credential management overhead. These are decisions made better by people who understand the attacker perspective.

As a compliance professional or GRC analyst, the vulnerability management framework — formal risk acceptance, compensating control documentation, remediation timelines tied to CVSS severity — maps directly to what PCI DSS, HIPAA, SOC 2, and ISO 27001 require. The analytical vocabulary (CVSS, EPSS, CVE, KEV) is the language that technical findings must be translated into for compliance documentation.

3.5.3 Module 3 Key Terms Reference

The following terms were covered in Module 3 and should be part of your professional vocabulary:

Active Reconnaissance — The phase of information gathering that directly interacts with target systems, generating network traffic and potentially triggering security alerts. Requires authorization before beginning.

ARP Poisoning — A technique that sends fake ARP replies to corrupt the ARP caches of communicating hosts, positioning the attacker as a man-in-the-middle who can capture and forward traffic between them.

Banner Grabbing — The practice of connecting to a network service to read its identification string, which often reveals software type, version, and operating system information.

CISA KEV (Known Exploited Vulnerabilities Catalog) — The U.S. government's catalog of CVEs confirmed to be actively exploited in the wild. The strongest available remediation prioritization signal.

Compensating Control — A security control implemented to reduce the risk of a vulnerability when direct remediation is not immediately possible.

CVE (Common Vulnerabilities and Exposures) — The global standard for identifying and naming specific security vulnerabilities. Format: CVE-[year]-[sequence].

CVSS (Common Vulnerability Scoring System) — The framework for assigning standardized severity scores to vulnerabilities, using Base, Temporal, and Environmental score groups.

CWE (Common Weakness Enumeration) — A classification system for the underlying code-level weakness types that cause vulnerabilities.

DNS Zone Transfer (AXFR) — A DNS mechanism that copies the complete zone file from a primary to a secondary DNS server. Misconfigured servers allow zone transfers to any requestor, potentially revealing the entire DNS namespace.

Enumeration — The process of extracting detailed, service-specific information from discovered open ports, beyond the port state and version information provided by scanning.

EPSS (Exploit Prediction Scoring System) — A machine learning model that predicts the probability a CVE will be exploited within the next 30 days, providing a real-world exploitation likelihood signal distinct from CVSS severity scores.

False Negative — When a vulnerability scanner fails to detect a vulnerability that actually exists. More dangerous than false positives because it creates false confidence.

False Positive — When a vulnerability scanner reports a vulnerability that does not actually exist or is not exploitable in the current environment.

Host Discovery — The process of determining which IP addresses in a target range have live hosts, before proceeding to port scanning. Nmap's -sn flag performs host discovery without port scanning.

MITRE ATT&CK — A knowledge base of adversary tactics, techniques, and procedures (TTPs) derived from real-world intrusion observations, used to understand what attackers do with vulnerabilities after exploitation.

NVD (National Vulnerability Database) — NIST's enrichment layer on the CVE system, providing CVSS scores, CWE classifications, CPE affected product data, and reference links for every CVE.

OSINT (Open-Source Intelligence) — Intelligence gathered from publicly available sources without direct interaction with the target's systems.

Passive Reconnaissance — Information gathering from public sources that generates zero traffic to the target, leaving no traces in target logs.

Pivoting — The technique of using one piece of intelligence to discover additional intelligence, systematically expanding the picture of the target's attack surface.

Port Scanning — The process of probing a range of port numbers on one or more hosts to determine which ports are open, closed, or filtered.

Risk Acceptance — The formal organizational decision to acknowledge a vulnerability and accept the residual risk without remediation, with explicit documentation and executive authorization.

Shodan — A search engine for internet-connected devices that indexes service banners from every accessible port on every IP address, enabling passive discovery of internet-facing systems by organization, technology, or vulnerability.

SYN Scan (-sS) — Nmap's default scan type when run with root privileges. Sends SYN packets and analyzes responses without completing the TCP three-way handshake, providing fast, reliable port state determination.

UDP Scan (-sU) — Nmap's UDP scanning mode. Significantly slower than TCP scanning due to UDP's connectionless nature and OS rate limiting of ICMP Port Unreachable responses, but critical for discovering high-value services like SNMP, DNS, NTP, and TFTP.

Vulnerability — A weakness in a system, application, or process that can be exploited to cause harm, defined formally as a weakness in computational logic that when exploited results in negative impact to confidentiality, integrity, or availability.

Vulnerability Scanning — Automated testing of systems against databases of known vulnerabilities to identify security weaknesses. Distinct from penetration testing: scanning identifies potential weaknesses, penetration testing confirms exploitability.

Final Note on Module 3 Completion

Module 3 has built the complete intelligence and scanning foundation for professional penetration testing. You now understand how to map an organization's external attack surface from entirely public sources before touching a single system, how to systematically discover and enumerate every service on a target network, how to assess every discovered service against known vulnerability databases, and how to investigate, analyze, and communicate each finding with professional rigor.

These capabilities chain directly into Module 4 (Exploitation) — where the intelligence gathered in Module 3 becomes the input for selecting, customizing, and executing attack techniques. Every successful exploit is preceded by successful reconnaissance. Every accurate exploitation attempt is informed by thorough scanning and analysis. The quality of what comes next is bounded by the quality of what was done in Module 3.

═══════════════════════════════════════════════════════════
MODULE 3 — INFORMATION GATHERING AND VULNERABILITY SCANNING
COMPLETE
═══════════════════════════════════════════════════════════

Table of Contents

3.0 Introduction

Module Overview: Information Gathering and Vulnerability Scanning

Why This Module is the Most Critical in the Entire Penetration Testing Process

The Reconnaissance-Attack Continuum

Module Topics at a Glance

3.1 Performing Passive Reconnaissance

3.1.1 Overview

Why Passive Reconnaissance Matters at the Senior Level

3.1.2 Active Reconnaissance vs. Passive Reconnaissance

Passive Reconnaissance — Defined

Active Reconnaissance — Defined

The Legal Distinction

The Passive-Active Spectrum

3.1.3 The OSINT Methodology — How Professionals Think

The Intelligence Cycle

OSINT Intelligence Requirements for Penetration Testing

The Pivot Model

3.1.4 OSINT Tools — The Complete Professional Arsenal

OSINT Framework

OSINT Combine

SMART — Start.me Aggregated Resource Tool

SpiderFoot

Recon-ng

Maltego

theHarvester

Sherlock

hacker.org

3.1.5 DNS Lookups — Deep Dive

DNS Fundamentals — What Every Senior Penetration Tester Must Know

Complete DNS Record Type Reference

3.1.6 DNS Reconnaissance — Advanced Techniques

Zone Transfer (AXFR)

DNS Brute Forcing

DNS History and Passive DNS

3.1.7 Identification of Technical and Administrative Contacts

WHOIS Contact Records

BGP Intelligence — Autonomous System Numbers

3.1.8 WHOIS Intelligence — Extracting Maximum Value

Domain Portfolio Discovery

Registrar Security Analysis

3.1.9 DNS Lookups — Lab-Level Practical Reference

Essential DNS Tools

DNS Reconnaissance Workflow — Complete Lab Procedure

Online DNS Reconnaissance Tools

3.1.10 Cloud vs. Self-Hosted Applications and Related Subdomains

Identifying Cloud-Hosted Infrastructure

Subdomain Enumeration — Professional Techniques

3.1.11 Social Media Scraping

LinkedIn — The Primary Corporate Intelligence Source

Twitter / X Intelligence

GitHub and Code Repository Intelligence

3.1.12 Employee Intelligence Gathering

Building the Target Employee Profile

Email Address Format Discovery

3.1.13 Cryptographic Flaws

Why Cryptographic Analysis Belongs in Passive Reconnaissance

SSL/TLS Protocol Weaknesses

Major TLS Vulnerabilities — Professional Reference

Certificate Weaknesses

3.1.14 Finding Information from SSL Certificates

Reading Certificate Information

Certificate Transparency (CT) Log Mining

Certificate Analysis Tools

3.1.15 Company Reputation and Security Posture

Threat Intelligence Platforms

Paste Sites and Dark Web Monitoring

Public Vulnerability Databases

Bug Bounty Program Analysis

3.1.16 File Metadata

What is File Metadata?

What Metadata Reveals to Penetration Testers

Metadata Extraction Tools

The Metadata Intelligence Workflow

3.1.17 Web Archiving, Caching, and Public Code Repositories

The Wayback Machine — Internet Archive

Google Cache

Common Crawl

Public Code Repositories — Beyond GitHub

robots.txt and sitemap.xml