Combining passive DNS sources, certificate transparency logs, and active bruteforcing creates a much clearer picture of an organization's external attack surface. A company may know about its main website and production API, but what about old staging environments, forgotten development servers, and subdomains created during previous projects?
These assets can remain accessible long after the original teams stop maintaining them.
Manual subdomain discovery is useful for small assessments, but it becomes difficult to maintain when an organization manages hundreds of domains. An automated asset pipeline helps security teams discover, validate, organize, and monitor these assets continuously.
This guide explains how to combine OWASP Amass, Subfinder, and ShuffleDNS into a repeatable reconnaissance workflow using Linux CLI commands and scheduled cron jobs.
1. Understanding the Subdomain Reconnaissance Pipeline
Subdomain reconnaissance is the process of identifying subdomains associated with an authorized root domain.
For example, an organization might have:
- www.example.com – Main website
- api.example.com – Application API
- dev.example.com – Development environment
- staging.example.com – Testing environment
- admin.example.com – Administrative portal
Not all these assets will appear in the company's documentation.
A useful pipeline combines three discovery methods:
Passive reconnaissance: Collects information from public sources without directly querying every discovered host. Sources include certificate transparency logs, passive DNS databases, and search engines.
Active DNS resolution: Checks whether discovered domain names currently resolve through DNS.
Active bruteforcing: Generates potential subdomain names using a wordlist and checks whether they resolve. This should only be performed against domains explicitly authorized for testing.
No individual source provides a complete inventory. Combining multiple sources improves discovery coverage.
2. Setting Up Amass, Subfinder and ShuffleDNS
Before building the pipeline, install the required tools on a Linux testing environment.
Subfinder collects subdomains from passive sources, while OWASP Amass provides broader attack-surface discovery capabilities. ShuffleDNS uses MassDNS to resolve candidate names and perform wordlist-based discovery with wildcard handling.
Install the tools:
go install github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
go install github.com/projectdiscovery/shuffledns/cmd/shuffledns@latest
Install OWASP Amass using its official installation instructions.
Check that the tools are available:
amass -version
subfinder -version
shuffledns -version
massdns -h
ShuffleDNS requires MassDNS, a valid resolver list, and a wordlist for bruteforce operations.
Create a working directory:
mkdir -p ~/subdomain-pipeline/{input,output,logs}
cd ~/subdomain-pipeline
Create an authorized target file:
echo "example.com" > input/domains.txt
Replace the example domain with an organization-owned domain or an explicitly authorized assessment target.
3. Combining Passive Discovery and Active DNS Enumeration
The first stage collects information from different sources.
Run Subfinder:
subfinder \
-d example.com \
-all \
-silent \
-o output/subfinder.txt
The -all option queries all supported passive sources, which may take longer than the default discovery mode.
Next, run Amass in passive mode:
amass enum \
-passive \
-d example.com \
-o output/amass.txt
These commands collect potential subdomains without requiring active wordlist-based enumeration.
Combine the results:
cat output/subfinder.txt output/amass.txt \
| sort -u \
> output/passive-assets.txt
Now validate the discovered names using ShuffleDNS:
shuffledns \
-d example.com \
-list output/passive-assets.txt \
-r resolvers.txt \
-mode resolve \
-o output/resolved-assets.txt
The resolver file should contain valid DNS resolver addresses.
For additional discovery, use an approved wordlist:
shuffledns \
-d example.com \
-w wordlist.txt \
-r resolvers.txt \
-mode bruteforce \
-sw \
-o output/bruteforce-assets.txt
The -sw option enables strict wildcard checking.
Finally, combine both result sets:
cat output/resolved-assets.txt \
output/bruteforce-assets.txt \
| sort -u \
> output/all-assets.txt
The resulting file becomes the initial asset inventory.
4. Automating the Pipeline with Shell Scripts and Cron
Running commands manually every day is inefficient. A scheduled pipeline makes asset discovery repeatable.
Create a script named run-pipeline.sh:
#!/usr/bin/env bash
set -Eeuo pipefail
DOMAIN="example.com"
BASE="$HOME/subdomain-pipeline"
DATE=$(date +%F)
OUT="$BASE/output/$DATE"
LOG="$BASE/logs"
mkdir -p "$OUT" "$LOG"
# Passive discovery
subfinder -d "$DOMAIN" -all -silent \
-o "$OUT/subfinder.txt"
amass enum -passive -d "$DOMAIN" \
-o "$OUT/amass.txt"
# Combine passive findings
cat "$OUT/subfinder.txt" "$OUT/amass.txt" \
| sort -u > "$OUT/passive.txt"
# Resolve discovered subdomains
shuffledns -d "$DOMAIN" \
-list "$OUT/passive.txt" \
-r "$BASE/resolvers.txt" \
-mode resolve \
-o "$OUT/resolved.txt"
# Perform authorized wordlist discovery
shuffledns -d "$DOMAIN" \
-w "$BASE/wordlist.txt" \
-r "$BASE/resolvers.txt" \
-mode bruteforce \
-sw \
-o "$OUT/bruteforce.txt"
# Build final inventory
cat "$OUT/resolved.txt" "$OUT/bruteforce.txt" \
| sort -u > "$OUT/all-assets.txt"
echo "Pipeline completed: $OUT"
Make the script executable:
chmod 700 run-pipeline.sh
The script assumes that the required tools, resolver list, and wordlist are already installed and configured.
Vulnerable scheduling example:
* * * * * bash ~/run-pipeline.sh
This configuration runs every minute. It can create unnecessary DNS traffic, overlapping processes, and excessive log output.
Remediated scheduling example:
0 2 * * 1 /bin/bash /home/user/subdomain-pipeline/run-pipeline.sh >> /home/user/subdomain-pipeline/logs/cron.log 2>&1
This schedules the pipeline once every Monday at 2 AM.
For production use, add a process lock, execution timeout, explicit target allowlist, and log rotation. Store API credentials securely rather than placing them directly in scripts.
5. Validating Findings, Remediating Risks and Maintaining the Asset Inventory
Discovery is only the first part of reconnaissance. Security teams must also determine which findings represent actual risks.
Consider a development subdomain that still resolves to an outdated cloud service.
Vulnerable configuration:
dev.example.com CNAME old-service.example-cloud.com
If the cloud resource has been deleted but the DNS record remains, the organization may have a dangling DNS reference.
This does not automatically mean the resource can be claimed. The cloud provider, resource state, ownership, and verification requirements must be checked.
Remediated configuration:
# Remove the obsolete CNAME record
# Or restore the intended service and verify ownership.
dev.example.com CNAME active-service.example-cloud.com
The correct action depends on whether the service is still required.
For every discovered asset, maintain an inventory containing:
- Subdomain name
- DNS record type
- Resolved IP or CNAME
- Discovery source
- First-seen and last-seen dates
- Business owner
- Environment classification
- Verification status
- Remediation status
Compare new results with previous scans to identify newly discovered assets, removed records, and unexpected DNS changes.
A simple comparison can identify new findings:
comm -13 \
previous-assets.txt \
current-assets.txt \
> newly-discovered.txt
For reliable comparisons, keep both input files sorted and normalized.
Teams should investigate unexpected changes rather than automatically treating every new hostname as a vulnerability.
A continuous asset pipeline helps organizations identify forgotten infrastructure, reduce exposure, and improve visibility across development and production environments.
Final takeaway: Automated subdomain reconnaissance is not just about collecting more hostnames. It is about maintaining an accurate, continuously updated inventory and ensuring every exposed asset has an owner and a remediation plan.
Top comments (0)