collynce

Posted on May 26

Sentinel Nexus: AI-Powered Threat Intelligence Platform

#devchallenge #brightdatachallenge #ai #webdata

This is a submission for the Bright Data Real-Time AI Agents Challenge

What I Built

I built a modular, agentic threat intelligence platform that demonstrates the full potential of AI agents with real-time web access:

Sentinel Nexus – An autonomous, LLM-powered threat intelligence system that continuously discovers, accesses, extracts, and interacts with security threats across the open web, dark web, and social media. It is architected for extensibility and real-world impact, solving the core challenges of speed, access, and accuracy in threat detection.

BrightData MCP Python – A custom MCP server implementation that enables seamless integration with Bright Data's infrastructure, providing robust browser automation, protection bypass, and structured data extraction capabilities specifically designed for threat intelligence gathering.

Core Problem Solved

Traditional threat intelligence platforms are slow, siloed, and struggle to access protected or dynamic sources. They rely on manual workflows, miss emerging threats, and suffer from high false positive rates.

Sentinel Nexus bridges this gap by combining:

Real-time web monitoring via Bright Data's MCP
Modular, agentic workflows orchestrated by Google ADK
Automated, LLM-driven extraction and analysis
Dynamic tool injection and robust error handling

Demo

Platform: dashboard

Screenshots

Repositories

Main Repo: sentinel-nexus
MCP Server: brightdata-mcp-python

Project Structure

Sentinel Nexus

sentinel/
├── backend/
│   ├── app/
│   │   ├── agents/
│   │   │   └── threat_analysis/
│   │   │       ├── __init__.py
│   │   │       ├── agent.py
│   │   │       ├── prompt.py
│   │   │       ├── prompts.py
│   │   │       ├── tools.py
│   │   │       ├── utils/
│   │   │       └── sub_agents/
│   │   │           ├── discoverer/
│   │   │           ├── monitor_social_media/
│   │   │           ├── scrape_website/
│   │   │           ├── search_news/
│   │   │           ├── synthesizer/
│   │   │           └── threat_analysis/
│   │   ├── services/
│   │   ├── api/
│   │   ├── core/
│   │   └── models/
│   ├── requirements.txt
│   └── ...
├── frontend/
│   ├── components/
│   ├── pages/
│   ├── layouts/
│   ├── assets/
│   ├── app.config.ts
│   ├── nuxt.config.ts
│   └── ...
└── ...

MCP Server

brightdata-mcp/
├── server.py
├── browser_tools.py
├── browser_session.py
├── requirements.txt
└── README.md

How I Used Bright Data's Infrastructure

Sentinel Nexus is architected to deeply leverage Bright Data's capabilities through its MCP server integration, enabling AI agents with comprehensive web access across all four key actions:

1. Discover

Agent: DiscovererAgent

Prompt: Uses the MCP-injected search_engine tool to perform real-time, multi-engine searches (Google, Bing, Yandex) for news, forums, and social media sources.

Workflow:

Analyze the query and context
Formulate targeted search strategies
Call search_engine for each category
Output structured, categorized discoveries for downstream agents

2. Access

Agents: ScrapeWebsiteAgent, SearchNewsAgent, MonitorSocialMediaAgent

Prompt: Instructed to use MCP-injected Web Unlocker and browser automation tools to bypass CAPTCHAs, anti-bot, geo-restrictions, and manage sessions.

Workflow:

Automatically select static or dynamic extraction based on site complexity
Use proxy rotation and session management for stealth
Handle forbidden/proxy errors gracefully

3. Extract

Agents: ScrapeWebsiteAgent, SearchNewsAgent, MonitorSocialMediaAgent

Prompt:

Selects the most appropriate MCP tool for the content type and site structure (scrape_as_markdown, scrape_as_html, web_data_x_posts, etc.)
Fallback logic: If one tool fails, try the next (static → dynamic extraction)
Outputs structured JSON with metadata about the extraction process

4. Interact

Agents: ScrapeWebsiteAgent, MonitorSocialMediaAgent

Prompt:

Uses browser automation tools for human-like interaction (scraping_browser_click, scraping_browser_type, etc.)
Handles infinite scroll, popups, dynamic content loading, and captures screenshots

Modular Agentic Pipeline (Google ADK + MCP)

Dynamic Tool Injection: All tools are injected at runtime via MCP, with async initialization and mapping (utils/mcp_init.py).
Agent Orchestration: Sequential and parallel agents coordinate discovery, collection, and analysis.
LLM-Driven Prompts: Each agent is guided by a specialized prompt for its role, instructing it to call the right MCP tool for the job.

Real-World Use Cases

SOC Teams: Real-time alerts, automated IOC extraction, and firewall updates
Threat Researchers: Dark web monitoring, actor tracking, and cross-source correlation
Security Vendors: Automated enrichment and reputation scoring for threat feeds

Conclusion

Every agent in Sentinel Nexus is guided by a specialized prompt that instructs it to call the right MCP tool for its role—whether discovering sources, accessing protected sites, extracting structured data, or interacting with dynamic content. This agentic, tool-driven workflow—powered by Google ADK and Bright Data MCP—delivers a robust, scalable, and intelligent threat intelligence platform that fully realizes the vision of the Bright Data AI Web Access Hackathon.

DEV Community