This is a submission for the Bright Data AI Web Access Hackathon

What I Built

I built the AI News Sentiment Analyzer, a real-time tool that discovers, accesses, extracts, and analyzes news articles about artificial intelligence from across the web. Using Bright Data's MCP server capabilities and Groq's powerful llama3-70b-8192 model, the system provides insights into how AI is portrayed in current media.

In today's rapidly evolving AI landscape, keeping track of public perception and media coverage is crucial for researchers, companies, and policymakers. However, manually monitoring numerous news sources is time-consuming and subjective. My solution automates this process by:

1. Finding relevant AI news content across diverse sources
2. Extracting meaningful information from complex web pages
3. Analyzing the sentiment of coverage objectively
4. Presenting insights in an accessible format

The application features a clean Streamlit interface where users can search for AI-related topics. It then displays a sentiment distribution of the articles found, showing positive, neutral, and negative coverage. For positive and neutral articles, it provides full details with links to the original sources. For negative articles, it only shows the titles without links, as specified in the requirements.

🛠️ For this project, I used uv to install and manage Python dependencies, ensuring a faster and more efficient experience.

🌐 I also used mcp-use, which provides the easiest way to interact with mcp-use servers using custom agents.

Demo

How I Used Bright Data's Infrastructure

Bright Data's MCP server was absolutely essential to this project's success. It enabled all four key actions required for effective real-time web interaction:

{
    "mcpServers": {
      "Bright Data": {
        "command": "npx",
        "args": ["@brightdata/mcp"],
        "env": {
          "API_TOKEN": "",
          "WEB_UNLOCKER_ZONE": "",
          "BROWSER_AUTH": ""
        }
      }
    }
  }

1. Discover

The application uses Bright Data's MCP server to discover relevant AI news content across the web by:

• Searching for recent news articles about specific AI topics
• Finding content across major news sites like UOL, G1, TechCrunch, CNN, and BBC
• Identifying relevant articles based on content and recency

2. Access

The MCP server enables the application to navigate through complex news websites that would typically block automated access, including:

• Handling paywalls and cookie consent forms
• Accessing content behind JavaScript rendering
• Navigating multi-page articles

3. Extract

Once the MCP server accesses the content, the application extracts structured data including:

• Article titles
• Publication dates
• URLs for reference
• Article summaries
• Full article content for sentiment analysis

4. Interact

The MCP server simulates human-like interaction with websites, enabling:

• Scrolling through infinite-scroll pages
• Clicking on "Read More" buttons
• Navigating pagination
• Handling dynamic content loading

Without Bright Data's MCP capabilities, this project would be limited to analyzing pre-collected datasets or simple RSS feeds, significantly reducing its value for real-time sentiment analysis.

Performance Improvements

Using Bright Data's real-time web data access dramatically improved the AI system's performance compared to traditional approaches in several key ways:

1. Freshness of Data

Traditional approaches rely on pre-collected datasets or APIs that may be hours or days old. With Bright Data's MCP server, the application accesses the most current news articles available, ensuring that sentiment analysis reflects the very latest media coverage.

2. Breadth of Sources

Most traditional approaches are limited to sources with accessible APIs or RSS feeds. Bright Data's MCP server allows the application to access any public news website, regardless of its technical implementation, dramatically increasing the diversity of sources analyzed.

3. Content Depth

Traditional scrapers often struggle with JavaScript-rendered content, paywalls, and complex site structures. Bright Data's MCP server enables extraction of complete article content rather than just headlines or summaries, providing much richer data for sentiment analysis.

4. Adaptability

News websites frequently change their structure, breaking traditional scrapers. Bright Data's MCP server handles these changes seamlessly, ensuring consistent data extraction over time without requiring constant maintenance.

5. Contextual Understanding

With access to full article content rather than just metadata, the llama3-70b-8192 model can perform much more nuanced sentiment analysis, understanding context, tone, and implications that would be missed by traditional approaches.

6. Real-World Applications

The improved performance enables practical applications that wouldn't be possible with traditional approaches, including:

• Corporate reputation management for AI companies
• Investment decision support for AI markets
• Academic research on media portrayal of AI
• Policy development based on public concerns
• Educational content development with current examples

By combining Bright Data's real-time web access capabilities with advanced AI models, this project demonstrates how AI systems can provide valuable insights into complex topics like AI media coverage, with performance that far exceeds what would be possible using traditional web scraping or API-based approaches.

My Github repository to run

vec21 / ai-news-sentiment-analyzer

Real-time sentiment analyzer for AI-related news using Bright Data MCP and Groq's llama3-70b-8192 model

AI News Sentiment Analyzer

Overview

This project is a real-time sentiment analyzer for AI-related news, leveraging Bright Data's MCP server capabilities and Groq's powerful llama3-70b-8192 model. The system discovers, accesses, extracts, and analyzes news articles about artificial intelligence, providing insights into how AI is portrayed in current media.

Key Features

Real-Time Web Data Collection

The application uses Bright Data's MCP server to:

• Discover: Find relevant AI news content across major news sites
• Access: Navigate through complex news websites
• Extract: Pull structured data including titles, URLs, dates, and summaries
• Interact: Engage with dynamic, JavaScript-rendered pages to extract content

Advanced Sentiment Analysis

• Utilizes Groq's llama3-70b-8192 model for nuanced sentiment analysis
• Classifies articles as positive, neutral, or negative based on content
• Provides fallback to NLTK for sentiment analysis when needed

User-Friendly Interface

• Clean Streamlit interface with search functionality
• Visual representation of sentiment distribution
• Expandable article details with direct links…

View on GitHub