Building a Malware Intelligence Platform: 13 Public Feeds, Knowledge Graph, and Zero Budget

I spent the last 6 months building MalwareIntel, an open CTI (Cyber Threat Intelligence) platform that aggregates data from 13 public feeds into a unified knowledge graph. Here is the architecture, what worked, and what almost broke everything.

The Problem

SOC analysts and threat hunters need context. When an alert fires on a SHA256 hash, they need to know: what malware family is this? What APT group uses it? What MITRE ATT&CK techniques does it employ? What are the defensive mitigations?

Commercial platforms (Recorded Future, ThreatConnect, Anomali) solve this for $30,000-100,000/year. Most small and mid-size security teams cannot afford that. They end up checking 5-10 different free tools manually.

MalwareIntel aggregates all of that into one platform with a unified data model, for free.

Architecture

13 Public Feeds
    |
    v
Celery Workers (scheduled ingestion)
    |
    v
Validation + Normalization Pipeline
    |
    v
PostgreSQL (Supabase) ──→ Qdrant (semantic search)
    |
    v
FastAPI REST API
    |
    v
Next.js Frontend (Cloudflare Pages)

Backend: FastAPI + Python 3.12, SQLAlchemy 2.0 async, Celery + Redis for scheduled ingestion.

Frontend: Next.js 14 App Router, TypeScript strict, Tailwind + shadcn/ui. Dark mode only (SOC analysts work in the dark).

Database: Supabase PostgreSQL with full-text search via GIN indexes. Qdrant for semantic search over malware family descriptions.

Hosting: Cloudflare Pages (frontend), Hetzner VPS (backend + workers), Supabase (managed Postgres).

The 13 Feeds

Feed	What it provides	Schedule
MITRE ATT&CK (TAXII)	TTPs, threat actors, software	Weekly
MITRE D3FEND	Defensive mitigations	Weekly
MalwareBazaar (abuse.ch)	Malware hashes + families	Every 6h
ThreatFox (abuse.ch)	C&C IOCs	Every 6h
URLhaus (abuse.ch)	Malware distribution URLs	Every 6h
Feodo Tracker (abuse.ch)	Botnet C&C IPs	Daily
OTX AlienVault	Pulses, IOCs, TTPs	Every 12h
MISP public feeds	IOCs, events, galaxies	Every 12h
CISA KEV	Exploited vulnerabilities	Daily
Ransomware.live	Ransomware victims + groups	Every 6h
ANY.RUN feeds	Sandbox behaviors	Daily
HoneyDB	Honeypot IOCs	Daily
GitHub (SigmaHQ)	Detection rules	Weekly

The Data Model

The core insight: everything in CTI is a graph. A malware family uses TTPs. TTPs have mitigations. Actors use families. Campaigns deploy families against targets.

MalwareFamily ──[uses]──────→ TTP (ATT&CK)
MalwareFamily ──[attributed]─→ ThreatActor
MalwareFamily ──[generates]──→ IOC (IP, hash, domain, URL)
MalwareFamily ──[exploits]───→ CVE
TTP ──[mitigated by]─────────→ D3FEND Mitigation
Campaign ──[uses]────────────→ MalwareFamily
Campaign ──[attributed]──────→ ThreatActor

We model this with 12 PostgreSQL tables and 5 M2M junction tables. Key tables:

• malware_families: 4,000+ families with type taxonomy (ransomware, RAT, infostealer, loader...)
• iocs: 1.7M+ indicators with type, confidence, severity, TLP classification
• ttps: Full MITRE ATT&CK matrix with subtechniques
• d3fend_mitigations: Defensive countermeasures mapped to TTPs

The Hardest Part: Normalization

Every feed has its own format. MalwareBazaar calls it "signature", MITRE calls it "software", Malpedia uses different naming. Emotet vs Heodo vs Geodo are the same family.

Our normalization pipeline:

1. Fetch raw data from feed API
2. Validate format (IPs via , hashes by length, domains via RFC 1035)
3. Reject private IPs (RFC 1918), localhost, future dates
4. Normalize family names against our canonical taxonomy
5. Deduplicate by unique index on
6. Link IOCs to families by matching signatures against aliases
7. Score confidence based on source reliability and corroboration count

@celery_app.task(bind=True, max_retries=3, rate_limit="10/m")
async def ingest_malwarebazaar(self, lookback_hours: int = 24):
    source = await get_or_create_source("MalwareBazaar")
    raw_data = await fetch_malwarebazaar_recent(lookback_hours)
    for sample in raw_data:
        validated = MalwareBazaarSample.model_validate(sample)
        normalized = normalize_to_ioc(validated)
        await ioc_repo.upsert(normalized, source_id=source.id)
        # NEVER download the binary, only metadata

The Knowledge Graph

The crown jewel. An interactive force-directed graph that visualizes relationships between families, actors, TTPs, IOCs, campaigns, and mitigations.

Built with Canvas API (not D3 force simulation, which is too slow for 500+ nodes). Custom physics engine:

• Node repulsion: F = 14000 / dist²
• Edge attraction: F = (dist - ideal) × 0.006 × strength
• Central gravity: F = pos × 0.0008
• Damping: 0.82 per frame

6 node types with semantic colors: purple (families), red (actors), green (TTPs), blue (IOCs), amber (campaigns), lime (mitigations).

Defense Kits

The most popular feature: downloadable defense kits per malware family. Each kit includes IOCs, YARA rules, Sigma rules, ATT&CK mappings, and D3FEND mitigations. One click to get everything you need to detect and defend against a specific threat.

4,251 families have defense kits available. Cobalt Strike alone generates a 621KB kit with 45 files.

What I Learned

1. Feed reliability varies wildly. abuse.ch feeds are rock solid. Some MISP feeds have 30% duplicate data. Build retry logic and dedup from day one.

2. Anti-prompt-injection in enrichment. When using LLMs to enrich family descriptions, IOC values can contain adversarial text. Sanitize everything before feeding to the model.

3. Redis cache saves your API. Without caching, the stats endpoints were hitting the database on every request. Redis with 5-minute TTL reduced DB load by 80%.

4. Spanish CTI content is an empty niche. 95% of threat intelligence is in English. The blog section with CTI articles in Spanish gets more organic traffic than any English content we tried.

Numbers (6 months in)

Metric	Value
Malware families indexed	4,251
IOCs in database	1.7M+
Public feeds integrated	13
Blog articles (ES)	36
Defense kits available	4,251
Monthly cost	~50 EUR
Revenue	Pre-revenue (subscription model launching)

Stack Summary

Layer	Tech
Frontend	Next.js 14, TypeScript, Tailwind, shadcn/ui
Backend	FastAPI, Python 3.12, SQLAlchemy 2.0 async
Workers	Celery + Redis
Database	Supabase PostgreSQL
Vector search	Qdrant
Hosting	Cloudflare Pages + Hetzner VPS
CI/CD	GitHub Actions (1027/1027 tests green)

Links

• MalwareIntel
• Defense Kit example
• Blog (Spanish CTI content)

---

Building in public. If you work in security and want to contribute feeds or integrations, reach out on LinkedIn.