DEV Community

Cover image for Building a Lightweight Financial News Sentiment Analyzer with Python and SerpApi
Ssebina CHARLES
Ssebina CHARLES

Posted on

Building a Lightweight Financial News Sentiment Analyzer with Python and SerpApi

#python #ai #web3 #marketing

Building a Lightweight Financial News Sentiment Analyzer with Python and SerpApi

Tags: python api finance sentimentanalysis beginners

Description:
Learn how to build a simple financial news sentiment analyzer in Python using SerpApi. The project retrieves recent news for assets such as Apple, Tesla, and Bitcoin, then classifies the language as bullish, bearish, or neutral using a lightweight lexicon-based approach.

## Introduction

Financial markets move quickly.

Every day, investors are exposed to thousands of headlines discussing earnings, company announcements, market movements, upgrades, downgrades, risks, and economic events.

The problem is not necessarily finding financial news.

The problem is processing large amounts of news quickly.

In this project, I built a small Python-based financial sentiment engine that retrieves recent financial news and automatically classifies each article as:

  • 🟢 Bullish
  • 🔴 Bearish
  • 🟡 Neutral

The project uses SerpApi to retrieve Google News results and Python to perform the sentiment classification.
if you dont have SerpApi account please visit https://serpapi.com to grap one

The goal is not to build a magical stock-price predictor.

Instead, this project demonstrates how a developer can combine:

Python
   ↓
SerpApi
   ↓
Google News
   ↓
Financial headlines
   ↓
Text processing
   ↓
Sentiment scoring
   ↓
Bullish / Bearish / Neutral
Enter fullscreen mode Exit fullscreen mode

This is a lightweight foundation that can later be extended into a much more advanced financial research system.

What We Are Building

The application accepts an asset such as:

Apple
Tesla
Bitcoin
Enter fullscreen mode Exit fullscreen mode

It then searches for recent financial news related to that asset.

For example:

Apple financial market news
Enter fullscreen mode Exit fullscreen mode

The returned articles are processed and the program extracts:

Title
Snippet
Source
Enter fullscreen mode Exit fullscreen mode

The title and snippet are combined and passed through a simple sentiment engine.

A simplified example might look like:

Headline:
Apple reports record growth in quarterly revenue

Detected terms:
record
growth

Result:
BULLISH 🟢
Enter fullscreen mode Exit fullscreen mode

Another headline might contain:

Apple shares fall after weak earnings outlook

Detected terms:
fall

Result:
BEARISH 🔴
Enter fullscreen mode Exit fullscreen mode

And an article without enough bullish or bearish language might produce:

NEUTRAL 🟡
Enter fullscreen mode Exit fullscreen mode

Project Architecture

The project is intentionally simple.

                  ┌─────────────────────┐
                  │      Python App     │
                  └──────────┬──────────┘
                             │
                             ▼
                  ┌─────────────────────┐
                  │      SerpApi        │
                  └──────────┬──────────┘
                             │
                             ▼
                  ┌─────────────────────┐
                  │    Google News      │
                  └──────────┬──────────┘
                             │
                             ▼
                  ┌─────────────────────┐
                  │  News Headlines     │
                  │  + Snippets        │
                  └──────────┬──────────┘
                             │
                             ▼
                  ┌─────────────────────┐
                  │ Sentiment Engine    │
                  └──────────┬──────────┘
                             │
                ┌────────────┼────────────┐
                ▼            ▼            ▼
             Bullish      Neutral      Bearish
               🟢            🟡            🔴
Enter fullscreen mode Exit fullscreen mode

The current implementation is contained in a single Python file, setiment_engine.py. It defines a FinancialSentimentEngine class, bullish and bearish token sets, a news retrieval method, a sentiment analyzer, and a pipeline runner.

Requirements

You need:

  • Python 3
  • A SerpApi account/API key
  • Internet access
  • A terminal
  • The serpapi Python package

The official SerpApi Python package can be installed with:

pip install serpapi
Enter fullscreen mode Exit fullscreen mode

The current SerpApi Python client is based around serpapi.Client, which is the approach used in this project.

Step 1: Create the Project

Create a project directory:

mkdir financial-sentiment
cd financial-sentiment
Enter fullscreen mode Exit fullscreen mode

Create a virtual environment:

python -m venv .venv
Enter fullscreen mode Exit fullscreen mode

Activate it on Windows:

.venv\Scripts\activate
Enter fullscreen mode Exit fullscreen mode

You should see something similar to:

(.venv) C:\Users\YourName\financial-sentiment>
Enter fullscreen mode Exit fullscreen mode

Step 2: Install SerpApi

Upgrade pip:

python -m pip install --upgrade pip
Enter fullscreen mode Exit fullscreen mode

Then install the official SerpApi package:

python -m pip install serpapi
Enter fullscreen mode Exit fullscreen mode

Verify the installation:

python -m pip show serpapi
Enter fullscreen mode Exit fullscreen mode

The important detail here is that the modern client uses:

import serpapi

client = serpapi.Client(api_key="...")
Enter fullscreen mode Exit fullscreen mode

rather than the older:

from serpapi import GoogleSearch
Enter fullscreen mode Exit fullscreen mode

This difference is important because using the wrong interface can result in:

ImportError: cannot import name 'GoogleSearch' from 'serpapi'
Enter fullscreen mode Exit fullscreen mode

The official SerpApi Python repository identifies serpapi.Client as the primary interface and recommends the serpapi package.

Step 3: Configure the API Key

Do not hard-code your API key into your Python source.

Instead, store it in an environment variable.

On Windows CMD:

set SERPAPI_API_KEY=YOUR_API_KEY
Enter fullscreen mode Exit fullscreen mode

For a persistent Windows environment variable:

setx SERPAPI_API_KEY "YOUR_API_KEY"
Enter fullscreen mode Exit fullscreen mode

After using setx, open a new terminal.

You can check whether the variable exists without printing the secret:

if defined SERPAPI_API_KEY (echo API key is configured) else (echo API key is missing)
Enter fullscreen mode Exit fullscreen mode

A good result is:

API key is configured
Enter fullscreen mode Exit fullscreen mode

Environment variables are also the approach recommended by the official SerpApi Python package documentation for keeping API keys separate from application code.

Step 4: The Python Application

Here is the complete implementation.

import os
import re
import sys
import time

import serpapi


class FinancialSentimentEngine:
    """Fetch financial news and classify its sentiment."""

    BULLISH_TOKENS = {
        "growth",
        "surge",
        "gain",
        "profit",
        "bullish",
        "higher",
        "record",
        "success",
        "buy",
        "upgrade",
    }

    BEARISH_TOKENS = {
        "drop",
        "fall",
        "loss",
        "crash",
        "bearish",
        "lower",
        "risk",
        "decline",
        "sell",
        "downgrade",
    }

    def __init__(self, api_key: str):
        if not api_key:
            raise ValueError("Missing SERPAPI_API_KEY.")

        self.client = serpapi.Client(api_key=api_key)

    def fetch_market_news(self, ticker: str) -> list:
        """Fetch recent financial news using SerpApi Google News."""

        print(
            f"[Engine] Querying financial news for ticker: {ticker}..."
        )

        search_parameters = {
            "engine": "google",
            "q": f"{ticker} financial market news",
            "tbm": "nws",
            "tbs": "qdr:w",
        }

        try:
            dictionary_results = self.client.search(search_parameters)

            if "error" in dictionary_results:
                print(
                    f"[Error] SerpApi returned an error: "
                    f"{dictionary_results['error']}"
                )
                return []

            return dictionary_results.get("news_results", [])

        except Exception as exc:
            print(f"[Error] Network query failed: {exc}")
            return []

    @classmethod
    def analyze_text_sentiment(cls, text: str) -> str:
        """Perform lightweight lexicon-based sentiment classification."""

        normalized_text = text.lower()

        bull_score = sum(
            1
            for token in cls.BULLISH_TOKENS
            if re.search(
                rf"\b{re.escape(token)}\b",
                normalized_text
            )
        )

        bear_score = sum(
            1
            for token in cls.BEARISH_TOKENS
            if re.search(
                rf"\b{re.escape(token)}\b",
                normalized_text
            )
        )

        if bull_score > bear_score:
            return "BULLISH 🟢"

        if bear_score > bull_score:
            return "BEARISH 🔴"

        return "NEUTRAL 🟡"

    def run_pipeline(self, ticker: str) -> None:
        """Fetch news and classify the first five articles."""

        news_articles = self.fetch_market_news(ticker)

        if not news_articles:
            print(
                f"[Warning] No news found for {ticker} "
                "within the selected time window."
            )
            return

        print(f"\n=== SENTIMENT REPORT: {ticker.upper()} ===")

        for index, article in enumerate(
            news_articles[:5],
            start=1
        ):
            title = article.get("title", "No title")
            snippet = article.get("snippet", "")

            source = article.get(
                "source",
                "Unknown source"
            )

            if isinstance(source, dict):
                source = source.get(
                    "name",
                    "Unknown source"
                )

            sentiment = self.analyze_text_sentiment(
                f"{title} {snippet}"
            )

            print(f"\n[{index}] Source: {source}")
            print(f"    Headline: {title}")
            print(f"    Sentiment: {sentiment}")


def main():
    api_key = os.getenv("SERPAPI_API_KEY")

    if not api_key:
        print(
            "[Abort] Set the SERPAPI_API_KEY "
            "environment variable first."
        )
        sys.exit(1)

    engine = FinancialSentimentEngine(api_key)

    assets = [
        "Apple",
        "Tesla",
        "Bitcoin",
    ]

    for index, asset in enumerate(assets):
        engine.run_pipeline(asset)

        print("-" * 60)

        if index < len(assets) - 1:
            time.sleep(1)


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

This implementation follows the structure of the uploaded source while correcting the SerpApi client usage and indentation problems that caused the earlier failure.

Step 5: Understanding the Sentiment Engine

The heart of the project is the sentiment classifier.

We define two groups of words.

Bullish words 

BULLISH_TOKENS = {
    "growth",
    "surge",
    "gain",
    "profit",
    "bullish",
    "higher",
    "record",
    "success",
    "buy",
    "upgrade",
}
Enter fullscreen mode Exit fullscreen mode

These terms generally represent positive market language.

Bearish words 

BEARISH_TOKENS = {
    "drop",
    "fall",
    "loss",
    "crash",
    "bearish",
    "lower",
    "risk",
    "decline",
    "sell",
    "downgrade",
}
Enter fullscreen mode Exit fullscreen mode

These represent negative market language.

The project then normalizes the article text:

normalized_text = text.lower()
Enter fullscreen mode Exit fullscreen mode

This prevents capitalization from affecting the matching.

For example:

Growth
growth
GROWTH
Enter fullscreen mode Exit fullscreen mode

all become:

growth
Enter fullscreen mode Exit fullscreen mode

Why Use Regular Expressions?

Instead of checking whether a token simply exists somewhere inside the text, the implementation uses word boundaries:

re.search(
    rf"\b{re.escape(token)}\b",
    normalized_text
)
Enter fullscreen mode Exit fullscreen mode

This helps reduce accidental partial matches.

For example, we want:

risk
Enter fullscreen mode Exit fullscreen mode

to match the word:

risk
Enter fullscreen mode Exit fullscreen mode

rather than accidentally matching unrelated words that merely contain the same character sequence.

Calculating the Scores

The bullish score is calculated with:

bull_score = sum(
    1
    for token in cls.BULLISH_TOKENS
    if re.search(
        rf"\b{re.escape(token)}\b",
        normalized_text
    )
)
Enter fullscreen mode Exit fullscreen mode

The bearish score works the same way:

bear_score = sum(
    1
    for token in cls.BEARISH_TOKENS
    if re.search(
        rf"\b{re.escape(token)}\b",
        normalized_text
    )
)
Enter fullscreen mode Exit fullscreen mode

The final decision is simple:

bullish score > bearish score
        ↓
     BULLISH

bearish score > bullish score
        ↓
     BEARISH

scores are equal
        ↓
     NEUTRAL
Enter fullscreen mode Exit fullscreen mode

The uploaded implementation follows exactly this three-way decision structure.

Step 6: Retrieving Financial News

The application creates a SerpApi search request:

search_parameters = {
    "engine": "google",
    "q": f"{ticker} financial market news",
    "tbm": "nws",
    "tbs": "qdr:w",
}
Enter fullscreen mode Exit fullscreen mode

The important pieces are:

engine 

"engine": "google"
Enter fullscreen mode Exit fullscreen mode

This specifies the Google search engine interface.

q 

"q": f"{ticker} financial market news"
Enter fullscreen mode Exit fullscreen mode

For:

Apple
Enter fullscreen mode Exit fullscreen mode

the query becomes:

Apple financial market news
Enter fullscreen mode Exit fullscreen mode

tbm 

"tbm": "nws"
Enter fullscreen mode Exit fullscreen mode

This is used to target Google News results.

tbs 

"tbs": "qdr:w"
Enter fullscreen mode Exit fullscreen mode

This asks for a recent time window of approximately one week.

SerpApi's client accepts Google search parameters through client.search(...), with results returned in a dictionary-like object.

Step 7: Extracting Articles

The application retrieves:

news_articles = self.fetch_market_news(ticker)
Enter fullscreen mode Exit fullscreen mode

Then it processes only the first five results:

for index, article in enumerate(
    news_articles[:5],
    start=1
):
Enter fullscreen mode Exit fullscreen mode

For every article, it extracts:

title = article.get("title", "No title")
snippet = article.get("snippet", "")
Enter fullscreen mode Exit fullscreen mode

It also handles the source field because the returned source data can be represented as a dictionary:

source = article.get(
    "source",
    "Unknown source"
)

if isinstance(source, dict):
    source = source.get(
        "name",
        "Unknown source"
    )
Enter fullscreen mode Exit fullscreen mode

Finally, the title and snippet are combined:

sentiment = self.analyze_text_sentiment(
    f"{title} {snippet}"
)
Enter fullscreen mode Exit fullscreen mode

This gives the sentiment engine more context than the title alone.

Step 8: Running the Pipeline

The main() function reads the API key:

api_key = os.getenv("SERPAPI_API_KEY")
Enter fullscreen mode Exit fullscreen mode

Then it creates the engine:

engine = FinancialSentimentEngine(api_key)
Enter fullscreen mode Exit fullscreen mode

The demonstration analyzes:

assets = [
    "Apple",
    "Tesla",
    "Bitcoin",
]
Enter fullscreen mode Exit fullscreen mode

The pipeline runs once for each asset.

Step 9: Run the Program

Save the program as:

setiment_engine.py
Enter fullscreen mode Exit fullscreen mode

Then run:

python setiment_engine.py
Enter fullscreen mode Exit fullscreen mode

You should see output similar to:

[Engine] Querying financial news for ticker: Apple...

=== SENTIMENT REPORT: APPLE ===

[1] Source: Example News
    Headline: Apple reports strong revenue growth
    Sentiment: BULLISH 🟢

[2] Source: Example News
    Headline: Apple faces concerns about lower demand
    Sentiment: BEARISH 🔴

------------------------------------------------------------
[Engine] Querying financial news for ticker: Tesla...

=== SENTIMENT REPORT: TESLA ===

[1] Source: Example News
    Headline: Tesla announces higher production
    Sentiment: BULLISH 🟢
Enter fullscreen mode Exit fullscreen mode

The exact articles and classifications will change because the program retrieves current search results at runtime.

What Makes This Project Useful?

At first glance, counting words might look too simple.

And it is simple.

But simple systems can be useful as a starting point.

This project demonstrates several important software engineering concepts:

API integration

The application communicates with an external service and consumes structured search results.

Environment-based configuration

The API credential is separated from the source code.

Object-oriented design

The financial engine is encapsulated in:

class FinancialSentimentEngine:
Enter fullscreen mode Exit fullscreen mode

Error handling

The application catches failed requests and handles missing search results.

Text processing

The application normalizes text and analyzes lexical features.

Pipeline architecture

The complete process is divided into stages:

Input
 ↓
Search
 ↓
Extraction
 ↓
Normalization
 ↓
Scoring
 ↓
Classification
 ↓
Output
Enter fullscreen mode Exit fullscreen mode

Important Limitation: This Is Not a Trading Signal

This is probably the most important part of the project.

The program is not predicting the future price of a stock or cryptocurrency.

It is also not a professional quantitative trading model.

It does not account for:

  • Market structure
  • Price action
  • Trading volume
  • Macroeconomic data
  • Earnings estimates
  • Technical indicators
  • Institutional positioning
  • Options activity
  • News credibility
  • Context
  • Negation
  • Sarcasm
  • Historical correlations

For example, consider:

Apple avoids a major revenue decline despite weaker demand.
Enter fullscreen mode Exit fullscreen mode

A simple lexical system may see:

decline
weaker
Enter fullscreen mode Exit fullscreen mode

and classify the article negatively, even though the overall meaning could be more nuanced.

That is one of the fundamental weaknesses of keyword-based sentiment analysis.

The system should therefore be viewed as a news-research utility, not as an automated investment decision maker.

How I Would Improve This Project

This project provides a foundation for something much more advanced.

The next version could introduce a proper NLP model instead of manually defined keywords.

For example:

                  Financial News
                        │
                        ▼
                Text Cleaning
                        │
                        ▼
               NLP Model / LLM
                        │
                        ▼
              Sentiment Score
                        │
            ┌───────────┼───────────┐
            ▼           ▼           ▼
         Bullish      Neutral     Bearish
Enter fullscreen mode Exit fullscreen mode

The output could become numerical:

{
  "asset": "Apple",
  "sentiment": "bullish",
  "score": 0.82
}
Enter fullscreen mode Exit fullscreen mode

instead of simply:

BULLISH 🟢
Enter fullscreen mode Exit fullscreen mode

Possible Future Features

The project could eventually become a complete financial research platform.

For example:

                    Financial Research Platform
                              │
          ┌───────────────────┼───────────────────┐
          ▼                   ▼                   ▼
       News API           Market Data          Social Data
          │                   │                   │
          └───────────────────┼───────────────────┘
                              ▼
                       NLP / AI Analysis
                              │
              ┌───────────────┼───────────────┐
              ▼               ▼               ▼
          Sentiment        Trends          Events
              │               │               │
              └───────────────┼───────────────┘
                              ▼
                       Research Dashboard
Enter fullscreen mode Exit fullscreen mode

Possible additions include:

  • Historical sentiment storage
  • PostgreSQL
  • REST APIs
  • FastAPI
  • Redis caching
  • Background workers
  • Market price APIs
  • Technical indicators
  • Sentiment charts
  • News deduplication
  • Source credibility scoring
  • LLM-based financial summarization
  • Portfolio monitoring
  • Web dashboards
  • Alerts

That would transform this from a small command-line experiment into a much more complete financial intelligence platform.

Security Notes

Never put your real API key inside:

api_key = "my-real-secret-key"
Enter fullscreen mode Exit fullscreen mode

Do not commit API keys to GitHub.

Do not put them in screenshots.

Do not publish them in DEV.to articles.

Use:

os.getenv("SERPAPI_API_KEY")
Enter fullscreen mode Exit fullscreen mode

and configure the key through your environment.

If a key is accidentally published, treat it as compromised and rotate it.

A Note About the Name

The file in this project is currently called:

setiment_engine.py
Enter fullscreen mode Exit fullscreen mode

The intended spelling is probably:

sentiment_engine.py
Enter fullscreen mode Exit fullscreen mode

The typo does not affect Python functionality, but I recommend renaming it before publishing the repository:

ren setiment_engine.py sentiment_engine.py
Enter fullscreen mode Exit fullscreen mode

Then run:

python sentiment_engine.py
Enter fullscreen mode Exit fullscreen mode

That will make the public project look more polished.

Project Structure

For the first version, the project can remain small:

financial-sentiment/
│
├── .venv/
│
├── sentiment_engine.py
│
├── .gitignore
│
└── README.md
Enter fullscreen mode Exit fullscreen mode

Your .gitignore should include:

.venv/
.env
__pycache__/
*.pyc
Enter fullscreen mode Exit fullscreen mode

Final Result

With only Python and SerpApi, we now have a working pipeline capable of:

Asset
  ↓
Financial News Search
  ↓
Recent Articles
  ↓
Headline + Snippet
  ↓
Lexicon Analysis
  ↓
Bullish / Bearish / Neutral
Enter fullscreen mode Exit fullscreen mode

It is small, understandable, and easy to extend.

More importantly, it demonstrates an important engineering principle:

You do not need to begin with an enormous AI system to build something useful.

Start with a clear pipeline.

Then improve each stage.

Conclusion

This project started as a simple question:

Can we automatically process recent financial news and get a quick indication of its overall tone?

Using Python and SerpApi, the answer is yes.

The current implementation is deliberately lightweight. It uses a manually defined financial vocabulary and compares bullish and bearish word counts.

That makes it easy to understand and easy to modify.

But it is only the beginning.

A future version could combine real market data, historical sentiment, machine learning, NLP, and large language models to build a far more sophisticated financial research system.

For now, this project provides a clean starting point for developers who want to learn how to connect Python to an external API and turn raw news data into a simple analytical output.

**The next step is not making the code bigger.

The next step is making the analysis smarter.**

Disclaimer

This project is for educational and research purposes only. The sentiment output is not financial advice and should not be used as the sole basis for investment or trading decisions.

The importance of the research

Am in a developing phase of building an autonomous AI trading quantum system for more visit https://github.com/ssebinacharles/quant-bot-command-center

Top comments (1)

Collapse
 
ssebina_charles_01 profile image
Ssebina CHARLES •

thanks for your support