DEV Community

Cover image for I Built an OSINT Aggregator That Queries 5 Threat Intel Sources in One Command
Kiell Tampubolon
Kiell Tampubolon

Posted on

I Built an OSINT Aggregator That Queries 5 Threat Intel Sources in One Command

#osint #cybersecurity #opensource #python

One night. One command. Full threat intelligence picture.

The Problem

Every time I needed to check if an IP or domain was malicious, I'd end up with 5 browser tabs open:

  • VirusTotal
  • AlienVault OTX
  • Shodan
  • NVD CVE Database
  • GitHub Security Advisories

Copy. Paste. Wait. Repeat. Cross-reference. Then make a decision.

This workflow is fine once. But when you're triaging multiple alerts during an incident, or doing OSINT research on a list of indicators — it's a massive time sink.

So I built a tool to automate it.

Introducing SentinelScout

SentinelScout is an open-source CLI tool that queries 5 threat intelligence sources simultaneously, then uses AI to correlate the results into a single, actionable threat score.

$ sentinelscout query suspicious-domain.xyz

Query: suspicious-domain.xyz (5 sources)

Source         Status  Severity  Result
──────────────────────────────────────────────
Virustotal      [+]     HIGH     67/94 engines flagged
Alienvault      [+]     HIGH     3 pulses found, tags: [apt] [phishing]
Shodan          [+]     INFO     4 open ports detected
Cve             [-]     LOW      No known CVEs found
Github Adv      [-]     LOW      No advisories found

Threat Score: 87/100 [CRITICAL]
Enter fullscreen mode Exit fullscreen mode

No browser tabs. No copy-pasting. Just answers.

How It Works

The tool is built on three pillars:

1. Async Python — Everything Runs in Parallel

All 5 sources are queried concurrently using Python's asyncio. Results come back in 2-4 seconds total, regardless of how slow any single source is.

async def _query(indicator: str, sources_filter: list[str] | None) -> AnalysisReport:
    all_sources: list[BaseSource] = [
        VirusTotalSource(),
        AlienVaultSource(),
        ShodanSource(),
        CVESource(),
        GitHubAdvSource(),
    ]
    if sources_filter:
        all_sources = [s for s in all_sources if s.source.value in sources_filter]

    tasks = [s.query(indicator) for s in all_sources]
    results = await asyncio.gather(*tasks)
    return AnalysisReport(indicator=indicator, sources=list(results))
Enter fullscreen mode Exit fullscreen mode

The asyncio.gather() call fires all 5 API requests simultaneously. If one source is slow, it doesn't block the others. Total wait time = slowest source, not sum of all sources.

2. Source Adapters — Pluggable Architecture

Each threat intel source has its own adapter class that handles:

  • API authentication
  • Request/response handling
  • Data normalization to a standard IOCResult model

Adding a new source is just creating a new class:

class MySource(BaseSource):
    name = "mysource"
    source = Source.MYSOURCE

    async def query(self, indicator: str) -> IOCResult:
        # your scraping logic here
        return IOCResult(...)
Enter fullscreen mode Exit fullscreen mode

Every adapter returns the same shape. The CLI doesn't care which source it came from — it just formats and displays it.

3. AI Correlation — Making Sense of It All

When you add an OpenAI API key, GPT-4o-mini reads all the raw results and generates:

  • A plain-English threat summary
  • A 0-100 threat score
  • A severity label (LOW / MEDIUM / HIGH / CRITICAL)

This is especially useful when different sources give conflicting signals — AI helps you make a judgment call instead of staring at contradictory data.

Supported Sources

Source Type API Required Free Tier
VirusTotal Domain/IP/Hash Yes 500 req/day
AlienVault OTX IP/Domain/Hash Yes 10k pulses/day
Shodan IP only Yes 100 queries/month
NVD CVE Vulnerabilities No Unlimited
GitHub Advisories Vulnerabilities No Unlimited

Two sources (CVE and GitHub) work out of the box with no API key needed.

Quick Start 

pip install sentinelscout
Enter fullscreen mode Exit fullscreen mode

Get free API keys:

Set your keys:

export VIRUSTOTAL_API_KEY=your_key
export ALIENVAULT_API_KEY=your_key
export SHODAN_API_KEY=your_key
export OPENAI_API_KEY=your_key  # optional
Enter fullscreen mode Exit fullscreen mode

Run it:

sentinelscout query suspicious-domain.xyz
sentinelscout sources   # check config status
sentinelscout --help
Enter fullscreen mode Exit fullscreen mode

Project Structure 

sentinelscout/
├── cli.py              # CLI entrypoint (Typer + Rich)
├── analyzer.py         # AI correlation engine (OpenAI)
├── config.py           # .env loader
├── models.py           # IOCResult, AnalysisReport dataclasses
├── sources/
│   ├── base.py         # BaseSource abstract class
│   ├── virustotal.py   # VirusTotal API client
│   ├── alienvault.py   # AlienVault OTX client
│   ├── shodan.py       # Shodan API client
│   ├── cve.py          # NVD CVE feed scraper (no key needed)
│   └── github.py       # GitHub Security Advisories (no key needed)
└── pyproject.toml      # package metadata
Enter fullscreen mode Exit fullscreen mode

What I Learned

Async Python is powerful. Once you wrap your head around asyncio.gather(), you start seeing parallelism everywhere. It's the difference between a tool that takes 15 seconds and one that takes 3.

Structured data models > raw dicts. Defining IOCResult and AnalysisReport as clean dataclasses made the entire codebase simpler. Every source adapter returns the same shape. No more ad-hoc dict handling.

The best projects come from personal pain. I didn't build SentinelScout to impress anyone. I built it because I was annoyed with my own workflow. That specific irritation is what kept me going through the 3-hour build session — and it's what makes the tool actually useful, not just impressive-looking.

What's Next

Ideas on the roadmap:

  • 🌐 Web UI (Gradio or Streamlit) for non-CLI users
  • 📊 JSON/CSV export for SIEM integration
  • 🤖 Telegram bot for on-the-go IOC queries
  • 🔗 More sources: GreyNoise, AbuseIPDB, ThreatFox

Get the Code

Full source code, installation instructions, and documentation:

👉 github.com/glatinone/sentinelscout

Star it if you find it useful. Issues and PRs welcome.

Built with Python 3.10+, Typer, Rich, and httpx. No paid dependencies — runs entirely on free-tier APIs.

Top comments (0)