Building a Real-Time News Aggregator and Trend Tracker with Python and Joffstrends Search API

In the modern era of artificial intelligence and large language models (LLMs), access to clean, structured, and real-time web search data is more critical than ever. Whether you are building an AI agent, a market intelligence dashboard, or a simple automated news aggregator, you need a search API that is fast, reliable, and cost-effective.

While major search engines offer APIs, they are often notoriously expensive, complex to set up, and gated behind heavy enterprise contracts. This is where the Joffstrends Search API shines. It provides a streamlined, developer-first interface to query the web and retrieve structured search results instantly.

In this comprehensive tutorial, we will walk through how to integrate the Joffstrends Search API with Python. We will build a practical, production-ready command-line News Aggregator and Trend Tracker script that fetches real-time search results, filters them, and exports them to a structured format (JSON) ready for analysis or LLM ingestion.

---

Why Joffstrends Search API?

Before we dive into the code, let’s look at why the Joffstrends Search API is an excellent choice for developers:

1. Simplicity: No complex OAuth flows or multi-layered configurations. You pass a query and your API key, and you get clean JSON back.
2. Affordability: With plans starting at just £4.99, it is highly accessible for indie hackers, students, and startups.
3. Speed: Optimized for rapid response times, making it ideal for real-time applications.
4. Structured Output: Returns clean, parsed search results (titles, links, snippets) that are easy to manipulate programmatically.

---

Prerequisites

To follow along with this tutorial, you will need:

• Python 3.7 or higher installed on your machine.
• A Joffstrends Search API Key. If you don't have one yet, you can obtain it instantly via Gumroad:
  • Starter Plan (£4.99 one-time, 7 days access): Get Starter Plan
  • Monthly Plan (£9.99/month, 1,000 searches/month): Get Monthly Plan
  • Annual Plan (£89.99/year, 1,000 searches/month): Get Annual Plan (Note: Your unique API key is emailed to you within approximately 1 minute of purchase.)

---

Setting Up Your Python Environment

First, let's create a new directory for our project and set up a virtual environment to keep our dependencies isolated.

mkdir joffstrends-news-aggregator
cd joffstrends-news-aggregator
python3 -m venv venv
source venv/bin/activate  # On Windows use: venv\Scripts\activate

Next, we need to install the requests library, which is the standard Python library for making HTTP requests. We will also install python-dotenv to securely manage our API key using environment variables.

pip install requests python-dotenv

Create a .env file in your root directory to store your API key securely:

JOFFSTRENDS_API_KEY=your_actual_api_key_here

---

Understanding the API Endpoint

The Joffstrends Search API is hosted at:
https://api.joffstrends.co.uk

To perform a search, we send a request to the search endpoint. Let's write a clean, robust Python wrapper to interact with this endpoint.

---

Writing the Python Script

Create a new file named aggregator.py and add the following code. This script defines a JoffstrendsClient class, handles API requests, parses the results, and implements our news aggregator logic.

import os
import json
import sys
from datetime import datetime
import requests
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

class JoffstrendsClient:
    """
    A simple Python client wrapper for the Joffstrends Search API.
    """
    BASE_URL = "https://api.joffstrends.co.uk"

    def __init__(self, api_key=None):
        # Retrieve the API key from parameters or environment variables
        self.api_key = api_key or os.getenv("JOFFSTRENDS_API_KEY")
        if not self.api_key:
            raise ValueError(
                "API Key not found. Please set JOFFSTRENDS_API_KEY in your .env file "
                "or pass it directly to the client."
            )

    def search(self, query):
        """
        Performs a search query using the Joffstrends Search API.
        """
        url = f"{self.BASE_URL}/search"
        params = {
            "q": query,
            "key": self.api_key
        }

        try:
            response = requests.get(url, params=params, timeout=10)

            # Check if the request was successful
            if response.status_code == 200:
                return response.json()
            elif response.status_code == 401:
                print("[Error] Unauthorized: Invalid API key.", file=sys.stderr)
                return None
            elif response.status_code == 429:
                print("[Error] Rate limit exceeded. Please check your plan limits.", file=sys.stderr)
                return None
            else:
                print(f"[Error] API returned status code {response.status_code}", file=sys.stderr)
                return None

        except requests.exceptions.RequestException as e:
            print(f"[Error] Network request failed: {e}", file=sys.stderr)
            return None

def save_results_to_json(query, results, filename=None):
    """
    Saves search results to a structured JSON file.
    """
    if not filename:
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        safe_query = "".join(c for c in query if c.isalnum() or c in (' ', '_')).rstrip()
        safe_query = safe_query.replace(' ', '_').lower()
        filename = f"search_{safe_query}_{timestamp}.json"

    data_to_save = {
        "query": query,
        "timestamp": datetime.now().isoformat(),
        "results": results
    }

    with open(filename, 'w', encoding='utf-8') as f:
        json.dump(data_to_save, f, indent=4, ensure_ascii=False)

    print(f"\n[Success] Results successfully saved to {filename}")
    return filename

def main():
    print("=" * 60)
    print(" Joffstrends Search API - Python News Aggregator & Trend Tracker")
    print("=" * 60)

    # Initialize the client
    try:
        client = JoffstrendsClient()
    except ValueError as e:
        print(f"[Error] {e}")
        sys.exit(1)

    # Prompt user for a search query
    query = input("\nEnter search query or trend topic (e.g., 'Artificial Intelligence news'): ").strip()
    if not query:
        print("[Error] Query cannot be empty.")
        sys.exit(1)

    print(f"\nFetching real-time search results for: '{query}'...")
    results = client.search(query)

    if not results:
        print("[Error] Failed to retrieve results.")
        sys.exit(1)

    # Display results in a clean, readable format
    print("\n" + "=" * 60)
    print(f" TOP RESULTS FOR: {query.upper()}")
    print("=" * 60)

    search_items = []
    if isinstance(results, list):
        search_items = results
    elif isinstance(results, dict):
        search_items = results.get("results", results.get("items", []))
        if not search_items and "title" in results:
            search_items = [results]

    if not search_items:
        print("No search results found or unexpected response format.")
        print("Raw Response:", json.dumps(results, indent=2))
        sys.exit(0)

    for idx, item in enumerate(search_items[:10], 1):
        title = item.get("title", "No Title")
        link = item.get("link", item.get("url", "No URL"))
        snippet = item.get("snippet", item.get("description", "No description available."))

        print(f"\n[{idx}] {title}")
        print(f"    URL:  {link}")
        print(f"    Info: {snippet}")
        print("-" * 60)

    # Ask the user if they want to export the data
    export_choice = input("\nWould you like to export these results to a JSON file? (y/n): ").strip().lower()
    if export_choice == 'y':
        save_results_to_json(query, search_items)

if __name__ == "__main__":
    main()

---

How the Code Works

Let's break down the key components of our script:

1. Environment Configuration: We use python-dotenv to load the JOFFSTRENDS_API_KEY from a .env file. This is a security best practice that prevents you from accidentally hardcoding your credentials and pushing them to public repositories like GitHub.
2. The JoffstrendsClient Class: This class encapsulates the API logic. It constructs the correct URL (https://api.joffstrends.co.uk/search) and appends the query parameters q (your search query) and key (your API key).
3. Robust Error Handling: The search method handles various HTTP status codes. It alerts the developer if the key is unauthorized (401), if they have hit their rate limits (429), or if a network error occurs.
4. Data Normalization: Different search endpoints can return slightly different JSON structures. The script gracefully checks for common keys like results or items to ensure it can parse the output reliably.
5. Structured Export: The save_results_to_json function formats the output with a timestamp and saves it as a clean JSON file. This file can be easily fed into other scripts, databases, or AI pipelines.

---

Running the Script

To run your new aggregator, simply execute the script from your terminal:

python aggregator.py

You will be prompted to enter a search query. Try something like "Python web scraping 2026" or "latest space exploration news". The script will query the Joffstrends Search API, display the top 10 results directly in your terminal, and offer to save them to a local JSON file.

---

Taking It Further: AI and LLM Integration

One of the most powerful use cases for the Joffstrends Search API is Retrieval-Augmented Generation (RAG). Because the API returns clean, structured JSON, you can easily feed these results directly into an LLM (such as OpenAI's GPT-4, Anthropic's Claude, or a local Ollama model) to generate automated summaries, newsletters, or intelligence reports.

Here is a quick conceptual example of how you can pass the output of our aggregator to an LLM:

# Conceptual example of LLM integration
import openai

def generate_summary(search_results):
    # Format the search results into a clean text prompt
    context = "\n\n".join([
        f"Title: {item['title']}\nSnippet: {item['snippet']}" 
        for item in search_results
    ])

    prompt = f"Based on the following search results, write a 3-paragraph summary of the latest trends:\n\n{context}"

    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

By combining the real-time search capabilities of Joffstrends with the synthesis power of LLMs, you can build incredibly sophisticated autonomous agents and automated research tools with less than 100 lines of code.

---

Get Started Today

The Joffstrends Search API is the perfect companion for developers who want fast, reliable, and straightforward web search access without the enterprise price tag.

Choose the plan that fits your project needs and get your API key instantly:

• Starter Plan (£4.99 one-time, 7 days access): Get Starter Plan
• Monthly Plan (£9.99/month, cancel anytime): Get Monthly Plan
• Annual Plan (£89.99/year, save ~3 months): Get Annual Plan

Happy coding! If you build something cool with the Joffstrends Search API, let us know in the comments below!