DEV Community: NewsDataHub

Understanding Pagination with NewsDataHub API

olga s — Tue, 17 Dec 2024 20:23:29 +0000

This guide explains how to paginate through the results when using NewsDataHub API.

NewsDataHub API is a service that provides news data through a RESTful API interface. It implements cursor-based pagination to efficiently handle large datasets⁠, allowing developers to retrieve news articles in manageable batches. Each response includes a set of articles where each article object contains details like title, description, publication date, source, content, keywords, topics, and sentiment analysis⁠. The API uses a cursor parameter for seamless navigation through results⁠, and provides comprehensive documentation for advanced features like search parameters and filtering options⁠.

For documentation visit: https://newsdatahub.com/docs

APIs typically return a limited amount of data in their responses since returning all results in a single request is usually impractical. Instead, they use pagination — a technique that splits data into separate pages or batches. This allows clients to retrieve one page at a time, accessing a manageable subset of the results.

When you make an initial request to the /news endpoint and receive the first batch of results, the shape of the response looks like this:

{
    "next_cursor": "VW93MzoqpzM0MzgzMQpqwDAwMDQ5LjA6MzA0NTM0Mjk1T0xHag==",
        "total_results": 910310,
        "per_page": 10,
        "data": [
            {
                "id": "4927167e-93f3-45d2-9c53-f1b8cdf2888f",
                "title": "Jail time for wage theft: New laws start January",
                "source_title": "Dynamic Business",
                "source_link": "https://dynamicbusiness.com",
                "article_link": "https://dynamicbusiness.com/topics/news/jail-time-for-wage-theft-new-laws-start-january.html",
                "keywords": [
                    "wage theft",
                    "criminalisation of wage theft",
                    "Australian businesses",
                    "payroll errors",
                    "underpayment laws"
                ],
                "topics": [
                    "law",
                    "employment",
                    "economy"
                ],
                "description": "Starting January 2025, deliberate wage theft will come with serious consequences for employers in Australia.",
                "pub_date": "2024-12-17T07:15:00",
                "creator": null,
                "content": "The criminalisation of wage theft from January 2025 will be a wake-up call for all Australian businesses. While deliberate underpayment has rightly drawn scrutiny, our research reveals that accidental payroll errors are alarmingly common, affecting nearly 60% of companies in the past two years. Matt Loop, VP and Head of Asia at Rippling Starting January 1, 2025, Australias workplace compliance landscape will change dramatically. Employers who deliberately underpay employees could face fines as high as AU$8. 25 million or up to 10 years in prison under new amendments to the Fair Work Act 2009 likely. Employers must act decisively to ensure compliance, as ignorance or unintentional errors wont shield them from civil or criminal consequences. Matt Loop, VP and Head of Asia at Rippling, says: The criminalisation of wage theft from January 2025 will be a wake-up call for all Australian businesses. While deliberate underpayment has rightly drawn scrutiny, our research reveals that accidental payroll errors are alarmingly common, affecting nearly 60% of companies in the past two years. Adding to the challenge, many SMEs still rely on fragmented, siloed systems to manage payroll. This not only complicates operations but significantly increases the risk of errors heightening the potential for non-compliance under the new laws. The urgency for businesses to modernise their approach cannot be overstated. Technology offers a practical solution, helping to streamline and automate processes, reduce human error, and ensure compliance. But this is about more than just avoiding penalties. Accurate and timely pay builds trust with employees, strengthens workplace morale, and fosters accountability. The message is clear: wage theft isnt just a financial risk anymoreits a criminal offense. Now is the time to ensure your business complies with Australias new workplace laws. Keep up to date with our stories on LinkedIn, Twitter, Facebook and Instagram.",
                "media_url": "https://backend.dynamicbusiness.com/wp-content/uploads/2024/12/db-3-4.jpg",
                "media_type": "image/jpeg",
                "media_description": null,
                "media_credit": null,
                "media_thumbnail": null,
                "language": "en",
                "sentiment": {
                    "pos": 0.083,
                    "neg": 0.12,
                    "neu": 0.796
                }
            },
        // more article objects
      ]
  }

Notice the first property in the JSON response - next_cursor. The value in next_cursor points to the start of the next page of results. When making the next request, you specify the cursor query parameter like this:

https://api.newsdatahub.com/v1/news?cursor=VW93MzoqpzM0MzgzMQpqwDAwMDQ5LjA6MzA0NTM0Mjk1T0xHag==

The easiest way to try out paginating through the results is via Postman, or a similar tool. Here is a short video demonstrating how to use cursor value to paginate through the results in Postman.

https://youtu.be/G7kkTwCPtCE

When the next_cursor value is null, it indicates that you have reached the end of the available results for your selected criteria.

Paginating through results with Python

Here is how to set up basic pagination through NewsDataHub API results using Python.

import requests

# Make sure to keep your API keys secure
# Use environment variables instead of hardcoding
API_KEY = 'your_api_key'
BASE_URL = 'https://api.newsdatahub.com/v1/news'

headers = {
    'X-Api-Key': API_KEY,
    'Accept': 'application/json',
    'User-Agent': 'Mozilla/5.0 Chrome/83.0.4103.97 Safari/537.36'
}

params = {}
cursor = None

# Limit to 5 pages to avoid rate limiting while demonstrating pagination

for _ in range(5):
    params['cursor'] = cursor

    try:
        response = requests.get(BASE_URL, headers=headers, params=params)
        response.raise_for_status()
        data = response.json()
    except (requests.HTTPError, ValueError) as e:
        print(f"There was an error when making the request: {e}")
        continue

    cursor = data.get('next_cursor')

    for article in data.get('data', []):
        print(article['title'])

    if cursor is None:
        print("No more results")
        break

Index based Pagination

Some APIs use index-based pagination to split results into discrete chunks. With this approach, APIs return a specific page of data—similar to a table of contents in a book, where each page number points to a specific section.

While index-based pagination is simpler to implement, it has several drawbacks. It struggles with real-time updates, can produce inconsistent results, and puts more strain on the database since retrieving each new page requires sequentially scanning through previous records.

We've covered the fundamentals of cursor-based pagination in the NewsDataHub API. For advanced features like search parameters and filtering options, please consult the complete API documentation at https://newsdatahub.com/docs.

Crypto News Aggregator using Typescript, Next.js, NewsDataHub and CoinGecko APIs

olga s — Tue, 17 Dec 2024 20:03:06 +0000

In this article, we're going to walk through building a simple but useful cryptocurrency news aggregator app that uses NewsDataHub and CoinGecko APIs. This article is aimed at a beginner-level developer—feel free to skip any sections if you think they don't add value to your learning experience.

You can also see what the final project code looks like here: https://github.com/newsdatahub/crypto-news-aggregator

You can see what the production version of this app looks like right here: https://newsdatahub.com/crypto

Let’s start off by by creating a fresh Next.js project with Typescript support.

npx create-next-app@latest crypto-news-aggregator --typescript

When prompted, select:

Yes for ESLint
No for Tailwind CSS (we'll use CSS modules)
No for src/ directory
Yes for App Router
No for Turbopack
No for customize import alias (we'll set this up manually)

cd into the project’s folder:

cd crypto-news-aggregator

Project Structure Setup

After initialization, let's create our project structure. I'll explain the purpose of each directory and file.

mkdir -p app/components/{news-feed,price-ticker} __tests__ types

At the end of this tutorial you should end up with the following structure.

crypto-news-aggregator/
├── __tests__/                    # Test files
│   ├── Home.test.tsx
│   ├── NewsCard.test.tsx
│   └── PriceTicker.test.tsx
├── app/                          # Next.js app directory
│   ├── components/               # React components
│   │   ├── news-feed/            # News-related components
│   │   │   ├── NewsCard.tsx
│   │   │   └── index.ts
│   │   └── price-ticker/         # Price ticker components
│   │       ├── PriceTicker.tsx
│   │       └── index.ts
│   ├── layout.tsx               # Root layout component
│   ├── page.module.css          # Styles for main page
│   └── page.tsx                 # Main page component
├── public/                      # Static assets
├── types/                       # TypeScript type definitions
│   ├── cache.ts
│   ├── crypto.ts
│   ├── env.d.ts
│   ├── index.ts
│     └── news.ts
├── .env.example                # Example environment variables
├── .env.local                  # Environment variables (gitignored)
├── .eslintrc.json              # ESLint configuration
├── .gitignore                  # Git ignore rules
├── eslint.config.mjs           # ESLint module configuration
├── jest.config.mjs             # Jest configuration
├── jest.setup.js               # Jest setup file
├── next-env.d.ts               # Next.js TypeScript declarations
├── next.config.js              # Next.js configuration
├── package-lock.json           # Locked dependency versions
├── package.json                # Project dependencies
├── README.md
├── tsconfig.json               # TypeScript configuration
└── types.d.ts                  # Global TypeScript declarations

But before we get there we are going to need to clean up the project directory a little bit and then create some files.

The files that can be safely deleted

app/globals.css (if you're using module.css files)
all .svg files (in /public directory)
README.md (delete or update, since this is the default one from create-next-app)

If your favicon.ico is in the app directory, consider moving it to the public folder. While the favicon can work in both locations, moving it to public/ follows conventional structure and makes asset locations more explicit.

Testing

We need to install several testing packages

npm install --save-dev @testing-library/react @testing-library/jest-dom jest jest-environment-jsdom

Let's understand what each package does:

@testing-library/react: Provides utilities for testing React components
@testing-library/jest-dom: Adds custom Jest matchers
jest: The main testing framework
jest-environment-jsdom: Simulates a browser environment for our tests

Create types.d.ts for testing type definitions

import '@testing-library/jest-dom';
declare global {
  namespace jest {
    interface Matchers<R> {
      toBeInTheDocument(): R;
    }
  }
  interface Window {
    fetch: jest.Mock;
  }
}
export {};

Now let's install TypeScript type definitions so that our code editor can understand Node.js, React, and Jest APIs, enabling autocomplete and catching type errors during development.

npm install --save-dev @types/node @types/react @types/jest

After installing the packages, we need to configure Jest. Create a jest.config.mjs file in your project root.

jest.config.mjs:

import nextJest from 'next/jest.js';

const createJestConfig = nextJest({
  dir: './',
});

export default createJestConfig({
  testEnvironment: 'jest-environment-jsdom',
  setupFilesAfterEnv: ['<rootDir>/jest.setup.js']
});

Create a jest.setup.js file to import the DOM matchers.

jest.setup.js:

import '@testing-library/jest-dom';

Finally, add these lines to run you test script to your package.json under scripts:

"test": "jest",
"test:watch": "jest --watch"

So it would look like this:

  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint",
    "test": "jest",
    "test:watch": "jest --watch"
  },

Now you can run tests using npm test or npm run test:watch for watch mode. But we don’t have any tests just yet, we will add a test shortly.

Getting Your NewsDataHub API Token

Let's walk through the process of getting your API token.

Visit NewsDataHub.com

Create Your Account (no credit card required)

Enter your email address into the sign-up form
You will need to check your email for the verification code
Once you verify your account, you will be taken to your dashboard where you can find your API key

Adding the API key to Your Project

Create a .env.example file in your project root to serve as a template for required environment variables

NEXT_PUBLIC_API_URL=https://api.newsdatahub.com/v1/news
NEXT_PUBLIC_API_TOKEN=your_token_here

Then run the following to copy .env.example template into .env.local where your actual configuration will be

cp .env.example .env.local

Replace `your_token_here` in `.env.local` with your NewsDataHub API token from your dashboard.

.env.example is committed to git as a template, while .env.local contains actual secrets and is gitignored.

We are going to use words “token” and “key” interchangeably when referring to the API key. Once you have your API token, add it to your project's .env.local file:

NEXT_PUBLIC_API_URL=https://api.newsdatahub.com/v1
NEXT_PUBLIC_API_TOKEN=your_newsdatahub_token

Configuration Files Setup and Overview

Let's set up essential configuration files. I'll explain each one's purpose and content:

Updating .gitignore

The .gitignore file was automatically created during project initialization. It tells Git which files and folders to exclude from version control.

Let's make sure our .env.local file is ignored by adding the following to .gitignore.

# Environment files
.env*.local

Setting up Environment Type Definitions

Create types/env.d.ts to provide TypeScript type definitions for our environment variables:

declare global {
  namespace NodeJS {
    interface ProcessEnv {
      NEXT_PUBLIC_API_URL: string;
      NEXT_PUBLIC_API_TOKEN: string;
    }
  }
}

export {};

This file tells TypeScript about our environment variables, enabling proper type checking when accessing process.env values and providing autocomplete suggestions. Without it, TypeScript would consider these variables to be of type any .

ESLint Setup

Add .eslintrc.json to enable Next.js's default linting rules for performance and best practices.

{
  "extends": [
    "next/core-web-vitals"
  ]
}

Now let’s add the project code

Create types/cache.ts.

Defines the structure for our client-side caching system, specifying how we store timestamps and news data

import { NewsItem } from ".";

export interface CacheData {
    timestamp: number;
    data: NewsItem[];
}

Create types/crypto.ts.
Defines cryptocurrency price data structure from the CoinCap API, including price, market cap, and 24h changes.

export type CoinData = {
  [key: string]: {
    usd: number;
    usd_market_cap: number;
    usd_24h_vol: number;
    usd_24h_change: number;
    last_updated_at: number;
  }
}

Create news.ts
Contains interfaces for news items from NewsDataHub API and props for our NewsCard component.

export interface NewsItem {
    id: string;
    title: string;
    article_link: string;
    description: string;
    pub_date: string;
  }

export interface NewsCardProps {
    index: number;
    item: NewsItem;
}

Create types/index.ts.
Central export point for all type definitions, enabling clean imports.

export * from './cache';
export * from './news';
export * from './crypto';

NewsCard and PriceTicker Component Implementation

Next, we implement our components and their styles. Each component should be in its respective directory.

NewsCard component

app/components/news-feed/NewsCard.tsx

import styles from './styles.module.css';
import { NewsCardProps } from '@/types';

export const NewsCard: React.FC<NewsCardProps> = ({index, item}) => {
    return (
        <div key={index} className={styles.newsCard}>
        <h2 className={styles.newsTitle}>{item.title}</h2>
        <p>{item.description.slice(0, 200)+"..."} Read more 
        <br/>
        <br/>
            <a href={item.article_link} target="_blank"  rel="noopener noreferrer" className={styles.newsLink}>
            {item.article_link}
            </a>    
        </p>

        <div className={styles.newsDate}>
          {new Date(item.pub_date).toLocaleDateString()}
        </div>
      </div>
    )
}

app/components/news-feed/styles.module.css

.newsCard {
  padding: 15px;
  border: 1px solid #ddd;
  border-radius: 4px;
  margin-bottom: 15px;
}

.newsTitle {
  margin: 0 0 10px 0;
  font-size: 1.2em;
  color: #2e009a;
  font-family: math;
}

.newsDate {
  color: #666;
  font-size: 0.9em;
  margin-top: 10px;
}

.newsLink {
  color:darkcyan;
}

.newsLink:hover {
  color: rgb(3, 79, 79);
  cursor: pointer;
  text-decoration: underline;
}

app/components/news-feed/index.tsx

export { NewsCard } from './NewsCard';

PriceTicker Component

app/components/price-ticker/PriceTicker.tsx

import { useState, useEffect } from 'react';
import styles from './styles.module.css';
import { CoinData } from '@/types';

export const  PriceTicker = () => {
  const [prices, setPrices] = useState<CoinData>({});
  const [error, setError] = useState<string | null>(null);
  const [loading, setLoading] = useState(true);

  const fetchPrices = async () => {
    try {
      const response = await fetch('https://api.coingecko.com/api/v3/simple/price?ids=bitcoin,ethereum,dogecoin&vs_currencies=usd&include_24hr_change=true');
      if (!response.ok) throw new Error('Failed to fetch prices');

      const data = await response.json();

      setPrices(data);
      setError(null);
    } catch (err) {
      setError('Failed to load prices');
      console.error('Price fetch error:', err);
    } finally {
      setLoading(false);
    }
  };

  useEffect(() => {
    fetchPrices();
    const interval = setInterval(fetchPrices, 60000); // Update every minute
    return () => clearInterval(interval);
  }, []);

  if (loading) return <div className={styles.ticker}>Loading prices...</div>;
  if (error) return <div className={styles.ticker}>Price data unavailable</div>;

  return (
    <div className={styles.ticker}>
      {Object.entries(prices).map(([coinId, data]) => {
        const price = data.usd.toLocaleString('en-US', {
          style: 'currency',
          currency: 'USD',
          minimumFractionDigits: 2,
          maximumFractionDigits: 2
        });

        const change = data.usd_24h_change || 0;
        const changeClass = change >= 0 ? styles.positive : styles.negative;

        return (
            <div key={coinId} className={styles.cryptoPrice}>
              <span className={styles.symbol}>{coinId.toUpperCase()}</span>
              <span className={styles.price}>{price}</span>
              <span className={`${styles.change} ${changeClass}`}>
       {change >= 0 ? '↑' : '↓'}
                {Math.abs(change).toFixed(2)}%
     </span>
            </div>
        );
      })}
    </div>
);
}

app/components/price-ticker/styles.module.css

.ticker {
  background: #1a1a1a;
  color: white;
  padding: 10px;
  border-radius: 8px;
  margin-bottom: 20px;
  overflow-x: auto;
  display: flex;
  gap: 20px;
  align-items: center;
}

.cryptoPrice {
  display: flex;
  align-items: center;
  gap: 8px;
  padding: 4px 8px;
  background: rgba(255, 255, 255, 0.1);
  border-radius: 4px;
  white-space: nowrap;
}

.symbol {
  font-weight: bold;
  color: #ffd700;
}

.price {
  font-family: monospace;
}

.change {
  font-size: 0.9em;
  padding: 2px 6px;
  border-radius: 4px;
}

.positive {
  color: #00ff00;
}

.negative {
  color: #ff4444;
}

@keyframes slide {
  0% {
    transform: translateX(100%);
  }
  100% {
    transform: translateX(-100%);
  }
}

app/components/news-feed/index.tsx

export { PriceTicker } from './PriceTicker';

Building the Main Page Component

In Next.js App Router, the main page of our application lives in app/page.tsx. While the file is going to be named page.tsx following Next.js conventions, we name our component Home to clearly indicate its purpose as our application's home page.

Go ahead and update app/components/page.tsx with the following code

'use client';

import { useState, useEffect } from 'react';
import { PriceTicker } from '@/app/components/price-ticker';
import { NewsCard } from '@/app/components/news-feed';
import { CacheData, NewsItem } from '@/types';
import styles from './page.module.css';

// Environment variables for API configuration
const API_URL = process.env.NEXT_PUBLIC_API_URL;
const API_TOKEN = process.env.NEXT_PUBLIC_API_TOKEN;

// Cache duration set to one hour
const CACHE_DURATION = 1000 * 60 * 60;
const TOPICS = ['cryptocurrency'];

// In-memory cache for storing news data
const cache: Record<string, CacheData> = {};

export default function Home() {
  // State management using React hooks
  const [news, setNews] = useState<NewsItem[]>([]);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const [lastUpdated, setLastUpdated] = useState<Date | null>(null);

  // Fetches news data with built-in caching
  const fetchNews = async (topics: string[]) => {
    const cacheKey = topics.sort().join(',');
    const cachedData = cache[cacheKey];

    if (cachedData && Date.now() - cachedData.timestamp < CACHE_DURATION) {
      setNews(cachedData.data);
      setLastUpdated(new Date(cachedData.timestamp));
      return;
    }

    setLoading(true);
    setError(null);

    try {
      const response = await fetch(`${API_URL}?language=en&topic=cryptocurrency`, {
        headers: {
          'x-api-key': API_TOKEN,
          'Content-Type': 'application/json'
        },
      });

      if (!response.ok) throw new Error('Failed to fetch news');

      const articles = await response.json();
      const data: NewsItem[] = articles.data;

      cache[cacheKey] = {
        timestamp: Date.now(),
        data
      };

      setNews(data);
      setLastUpdated(new Date());
    } catch (err) {
      setError(err instanceof Error ? err.message : 'An error occurred');
    } finally {
      setLoading(false);
    }
  };

  // Fetch data when component mounts
  useEffect(() => {
    fetchNews(TOPICS);
  }, []);

  // Handler for manual refresh
  const handleRefresh = () => {
    const cacheKey = TOPICS.sort().join(',');
    delete cache[cacheKey];
    fetchNews(TOPICS);
  };

  return (
    <div className={styles.container}>
      <PriceTicker />

      {lastUpdated && (
        <div className={styles.lastUpdated}>
          Last updated: {lastUpdated.toLocaleTimeString()}
          <button onClick={handleRefresh} className={styles.refreshButton}>
            Refresh
          </button>
        </div>
      )}

      {error && <div className={styles.error}>{error}</div>}

      {loading ? (
        <div className={styles.loading}>Loading...</div>
      ) : (
        news.map((item: NewsItem, index: number) => (
          <NewsCard key={item.id || index} item={item} index={index} />
        ))
      )}
    </div>
  );
}

Update page.module.css with the following styles:

.container {
  max-width: 800px;
  margin: 0 auto;
  padding: 20px;
}

.topics {
  margin-bottom: 20px;
}

.topic {
  margin-right: 10px;
  padding: 5px 10px;
  border: 1px solid #ddd;
  border-radius: 4px;
  background: none;
  cursor: pointer;
}

.topicSelected {
  background: #007bff;
  color: white;
  border-color: #007bff;
}

.error {
  color: #dc3545;
  padding: 10px;
  border: 1px solid #dc3545;
  border-radius: 4px;
  margin-bottom: 15px;
}

.loading {
  text-align: center;
  padding: 20px;
}

.lastUpdated {
  color: #666;
  font-size: 0.9em;
  margin-bottom: 15px;
}

.refreshButton {
  background: #007bff;
  color: white;
  border: none;
  padding: 5px 10px;
  border-radius: 4px;
  cursor: pointer;
  margin-left: 10px;
}

Update app/layout.tsx with the following:

export default function RootLayout({
  children,
}: {
  children: React.ReactNode
}) {
  return (
    <html lang="en">
      <head>
        <title>Crypto News Aggregator Application</title>
        <meta name="description" content="Crypto News Aggregator Application" />
      </head>
      <body>
        {children}
      </body>
    </html>
  )
}

Running your project

Go ahead and run your project

npm run dev

You can find your running app at https://localhost:3000

Congrats on finishing the project! 🏆 👏

Testing the Page Component

We are going to add a test for the Home component that verifies that it correctly renders news content after fetching data. Go ahead and create this test file.

__tests__/Home.test.tsx


import { render, screen, waitFor } from '@testing-library/react';
import Home from '@/app/page';

describe('Home', () => {
  beforeEach(() => {
    // Set up test environment variables
    process.env.NEXT_PUBLIC_API_URL = 'http://test-api.com';
    process.env.NEXT_PUBLIC_API_TOKEN = 'test-token';

    // Mock fetch for both API endpoints
    global.fetch = jest.fn((url) => {
      // Mock responses for different API calls
      if (url.includes('api.coingecko.com')) {
        return Promise.resolve({
          ok: true,
          json: () => Promise.resolve({
            bitcoin: { usd: 65000, usd_24h_change: 2.5 }
          }),
          status: 200,
        } as Response);
      }
      return Promise.resolve({
        ok: true,
        json: () => Promise.resolve({
          data: [{
            id: '1',
            title: 'News Title',
            description: 'News Description',
            url: 'https://test.com',
            published_at: '2024-03-25'
          }]
        }),
        status: 200,
      } as Response);
    }) as jest.Mock;
  });

  test('renders news feed', async () => {
    render(<Home />);
    await waitFor(
      () => expect(screen.getByText("News Title")).toBeInTheDocument(),
      { timeout: 3000 }
    );
  });
});

Run the test

npm run test

This test verifies that our Home component successfully renders news content after fetching data.

Additional tests for the PriceTicker and NewsCard components can be found in the project's GitHub repository. These tests cover basic component-specific functionality and rendering behavior. I encourage you to create more tests for this project.

Consider improving this project further.

Some ideas:

Implement proper loading states
Add pagination for news items
Implement more sophisticated caching
Enhance the testing suite
You can change the topic query param to fetch different types of news

Thanks for following along! 😄

Cover image credit: Photo by RDNE Stock project: https://www.pexels.com/photo/selective-focus-photo-of-silver-and-gold-bitcoins-8369648/

Crypto News Aggregator using Typescript, Next.js, NewsDataHub and CoinGecko APIs

olga s — Tue, 17 Dec 2024 15:42:54 +0000

You can also see what the final project code looks like here: https://github.com/newsdatahub/crypto-news-aggregator

You can see what the production version of this app looks like right here: https://newsdatahub.com/crypto

Let’s start off by by creating a fresh Next.js project with Typescript support.

npx create-next-app@latest crypto-news-aggregator --typescript

When prompted, select:

Yes for ESLint
No for Tailwind CSS (we'll use CSS modules)
No for src/ directory
Yes for App Router
No for Turbopack
No for customize import alias (we'll set this up manually)

cd into the project’s folder:

cd crypto-news-aggregator

Project Structure Setup

After initialization, let's create our project structure. I'll explain the purpose of each directory and file.

mkdir -p app/components/{news-feed,price-ticker} __tests__ types

At the end of this tutorial you should end up with the following structure.

crypto-news-aggregator/
├── __tests__/                    # Test files
│   ├── Home.test.tsx
│   ├── NewsCard.test.tsx
│   └── PriceTicker.test.tsx
├── app/                          # Next.js app directory
│   ├── components/               # React components
│   │   ├── news-feed/            # News-related components
│   │   │   ├── NewsCard.tsx
│   │   │   └── index.ts
│   │   └── price-ticker/         # Price ticker components
│   │       ├── PriceTicker.tsx
│   │       └── index.ts
│   ├── layout.tsx               # Root layout component
│   ├── page.module.css          # Styles for main page
│   └── page.tsx                 # Main page component
├── public/                      # Static assets
├── types/                       # TypeScript type definitions
│   ├── cache.ts
│   ├── crypto.ts
│   ├── env.d.ts
│   ├── index.ts
│     └── news.ts
├── .env.example                # Example environment variables
├── .env.local                  # Environment variables (gitignored)
├── .eslintrc.json              # ESLint configuration
├── .gitignore                  # Git ignore rules
├── eslint.config.mjs           # ESLint module configuration
├── jest.config.mjs             # Jest configuration
├── jest.setup.js               # Jest setup file
├── next-env.d.ts               # Next.js TypeScript declarations
├── next.config.js              # Next.js configuration
├── package-lock.json           # Locked dependency versions
├── package.json                # Project dependencies
├── README.md
├── tsconfig.json               # TypeScript configuration
└── types.d.ts                  # Global TypeScript declarations

But before we get there we are going to need to clean up the project directory a little bit and then create some files.

The files that can be safely deleted

app/globals.css (if you're using module.css files)
all .svg files (in /public directory)
README.md (delete or update, since this is the default one from create-next-app)

Testing

We need to install several testing packages

npm install --save-dev @testing-library/react @testing-library/jest-dom jest jest-environment-jsdom

Let's understand what each package does:

@testing-library/react: Provides utilities for testing React components
@testing-library/jest-dom: Adds custom Jest matchers
jest: The main testing framework
jest-environment-jsdom: Simulates a browser environment for our tests

Create types.d.ts for testing type definitions

import '@testing-library/jest-dom';
declare global {
  namespace jest {
    interface Matchers<R> {
      toBeInTheDocument(): R;
    }
  }
  interface Window {
    fetch: jest.Mock;
  }
}
export {};

Now let's install TypeScript type definitions so that our code editor can understand Node.js, React, and Jest APIs, enabling autocomplete and catching type errors during development.

npm install --save-dev @types/node @types/react @types/jest

After installing the packages, we need to configure Jest. Create a jest.config.mjs file in your project root.

jest.config.mjs:

import nextJest from 'next/jest.js';

const createJestConfig = nextJest({
  dir: './',
});

export default createJestConfig({
  testEnvironment: 'jest-environment-jsdom',
  setupFilesAfterEnv: ['<rootDir>/jest.setup.js']
});

Create a jest.setup.js file to import the DOM matchers.

jest.setup.js:

import '@testing-library/jest-dom';

Finally, add these lines to run you test script to your package.json under scripts:

"test": "jest",
"test:watch": "jest --watch"

So it would look like this:

  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint",
    "test": "jest",
    "test:watch": "jest --watch"
  },

Now you can run tests using npm test or npm run test:watch for watch mode. But we don’t have any tests just yet, we will add a test shortly.

Getting Your NewsDataHub API Token

Let's walk through the process of getting your API token.

Visit NewsDataHub.com

Create Your Account (no credit card required)

Enter your email address into the sign-up form
You will need to check your email for the verification code
Once you verify your account, you will be taken to your dashboard where you can find your API key

Adding the API key to Your Project

Create a .env.example file in your project root to serve as a template for required environment variables

NEXT_PUBLIC_API_URL=https://api.newsdatahub.com/v1/news
NEXT_PUBLIC_API_TOKEN=your_token_here

Then run the following to copy .env.example template into .env.local where your actual configuration will be

cp .env.example .env.local

Replace `your_token_here` in `.env.local` with your NewsDataHub API token from your dashboard.

.env.example is committed to git as a template, while .env.local contains actual secrets and is gitignored.

We are going to use words “token” and “key” interchangeably when referring to the API key. Once you have your API token, add it to your project's .env.local file:

NEXT_PUBLIC_API_URL=https://api.newsdatahub.com/v1
NEXT_PUBLIC_API_TOKEN=your_newsdatahub_token

Configuration Files Setup and Overview

Let's set up essential configuration files. I'll explain each one's purpose and content:

Updating .gitignore

The .gitignore file was automatically created during project initialization. It tells Git which files and folders to exclude from version control.

Let's make sure our .env.local file is ignored by adding the following to .gitignore.

# Environment files
.env*.local

Setting up Environment Type Definitions

Create types/env.d.ts to provide TypeScript type definitions for our environment variables:

declare global {
  namespace NodeJS {
    interface ProcessEnv {
      NEXT_PUBLIC_API_URL: string;
      NEXT_PUBLIC_API_TOKEN: string;
    }
  }
}

export {};

ESLint Setup

Add .eslintrc.json to enable Next.js's default linting rules for performance and best practices.

{
  "extends": [
    "next/core-web-vitals"
  ]
}

Now let’s add the project code

Create types/cache.ts.

Defines the structure for our client-side caching system, specifying how we store timestamps and news data

import { NewsItem } from ".";

export interface CacheData {
    timestamp: number;
    data: NewsItem[];
}

Create types/crypto.ts.
Defines cryptocurrency price data structure from the CoinCap API, including price, market cap, and 24h changes.

export type CoinData = {
  [key: string]: {
    usd: number;
    usd_market_cap: number;
    usd_24h_vol: number;
    usd_24h_change: number;
    last_updated_at: number;
  }
}

Create news.ts
Contains interfaces for news items from NewsDataHub API and props for our NewsCard component.

export interface NewsItem {
    id: string;
    title: string;
    article_link: string;
    description: string;
    pub_date: string;
  }

export interface NewsCardProps {
    index: number;
    item: NewsItem;
}

Create types/index.ts.
Central export point for all type definitions, enabling clean imports.

export * from './cache';
export * from './news';
export * from './crypto';

NewsCard and PriceTicker Component Implementation

Next, we implement our components and their styles. Each component should be in its respective directory.

NewsCard component

app/components/news-feed/NewsCard.tsx

import styles from './styles.module.css';
import { NewsCardProps } from '@/types';

export const NewsCard: React.FC<NewsCardProps> = ({index, item}) => {
    return (
        <div key={index} className={styles.newsCard}>
        <h2 className={styles.newsTitle}>{item.title}</h2>
        <p>{item.description.slice(0, 200)+"..."} Read more 
        <br/>
        <br/>
            <a href={item.article_link} target="_blank"  rel="noopener noreferrer" className={styles.newsLink}>
            {item.article_link}
            </a>    
        </p>

        <div className={styles.newsDate}>
          {new Date(item.pub_date).toLocaleDateString()}
        </div>
      </div>
    )
}

app/components/news-feed/styles.module.css

.newsCard {
  padding: 15px;
  border: 1px solid #ddd;
  border-radius: 4px;
  margin-bottom: 15px;
}

.newsTitle {
  margin: 0 0 10px 0;
  font-size: 1.2em;
  color: #2e009a;
  font-family: math;
}

.newsDate {
  color: #666;
  font-size: 0.9em;
  margin-top: 10px;
}

.newsLink {
  color:darkcyan;
}

.newsLink:hover {
  color: rgb(3, 79, 79);
  cursor: pointer;
  text-decoration: underline;
}

app/components/news-feed/index.tsx

export { NewsCard } from './NewsCard';

PriceTicker Component

app/components/price-ticker/PriceTicker.tsx

import { useState, useEffect } from 'react';
import styles from './styles.module.css';
import { CoinData } from '@/types';

export const  PriceTicker = () => {
  const [prices, setPrices] = useState<CoinData>({});
  const [error, setError] = useState<string | null>(null);
  const [loading, setLoading] = useState(true);

  const fetchPrices = async () => {
    try {
      const response = await fetch('https://api.coingecko.com/api/v3/simple/price?ids=bitcoin,ethereum,dogecoin&vs_currencies=usd&include_24hr_change=true');
      if (!response.ok) throw new Error('Failed to fetch prices');

      const data = await response.json();

      setPrices(data);
      setError(null);
    } catch (err) {
      setError('Failed to load prices');
      console.error('Price fetch error:', err);
    } finally {
      setLoading(false);
    }
  };

  useEffect(() => {
    fetchPrices();
    const interval = setInterval(fetchPrices, 60000); // Update every minute
    return () => clearInterval(interval);
  }, []);

  if (loading) return <div className={styles.ticker}>Loading prices...</div>;
  if (error) return <div className={styles.ticker}>Price data unavailable</div>;

  return (
    <div className={styles.ticker}>
      {Object.entries(prices).map(([coinId, data]) => {
        const price = data.usd.toLocaleString('en-US', {
          style: 'currency',
          currency: 'USD',
          minimumFractionDigits: 2,
          maximumFractionDigits: 2
        });

        const change = data.usd_24h_change || 0;
        const changeClass = change >= 0 ? styles.positive : styles.negative;

        return (
            <div key={coinId} className={styles.cryptoPrice}>
              <span className={styles.symbol}>{coinId.toUpperCase()}</span>
              <span className={styles.price}>{price}</span>
              <span className={`${styles.change} ${changeClass}`}>
       {change >= 0 ? '↑' : '↓'}
                {Math.abs(change).toFixed(2)}%
     </span>
            </div>
        );
      })}
    </div>
);
}

app/components/price-ticker/styles.module.css

.ticker {
  background: #1a1a1a;
  color: white;
  padding: 10px;
  border-radius: 8px;
  margin-bottom: 20px;
  overflow-x: auto;
  display: flex;
  gap: 20px;
  align-items: center;
}

.cryptoPrice {
  display: flex;
  align-items: center;
  gap: 8px;
  padding: 4px 8px;
  background: rgba(255, 255, 255, 0.1);
  border-radius: 4px;
  white-space: nowrap;
}

.symbol {
  font-weight: bold;
  color: #ffd700;
}

.price {
  font-family: monospace;
}

.change {
  font-size: 0.9em;
  padding: 2px 6px;
  border-radius: 4px;
}

.positive {
  color: #00ff00;
}

.negative {
  color: #ff4444;
}

@keyframes slide {
  0% {
    transform: translateX(100%);
  }
  100% {
    transform: translateX(-100%);
  }
}

app/components/news-feed/index.tsx

export { PriceTicker } from './PriceTicker';

Building the Main Page Component

Go ahead and update app/components/page.tsx with the following code

'use client';

import { useState, useEffect } from 'react';
import { PriceTicker } from '@/app/components/price-ticker';
import { NewsCard } from '@/app/components/news-feed';
import { CacheData, NewsItem } from '@/types';
import styles from './page.module.css';

// Environment variables for API configuration
const API_URL = process.env.NEXT_PUBLIC_API_URL;
const API_TOKEN = process.env.NEXT_PUBLIC_API_TOKEN;

// Cache duration set to one hour
const CACHE_DURATION = 1000 * 60 * 60;
const TOPICS = ['cryptocurrency'];

// In-memory cache for storing news data
const cache: Record<string, CacheData> = {};

export default function Home() {
  // State management using React hooks
  const [news, setNews] = useState<NewsItem[]>([]);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const [lastUpdated, setLastUpdated] = useState<Date | null>(null);

  // Fetches news data with built-in caching
  const fetchNews = async (topics: string[]) => {
    const cacheKey = topics.sort().join(',');
    const cachedData = cache[cacheKey];

    if (cachedData && Date.now() - cachedData.timestamp < CACHE_DURATION) {
      setNews(cachedData.data);
      setLastUpdated(new Date(cachedData.timestamp));
      return;
    }

    setLoading(true);
    setError(null);

    try {
      const response = await fetch(`${API_URL}?language=en&topic=cryptocurrency`, {
        headers: {
          'x-api-key': API_TOKEN,
          'Content-Type': 'application/json'
        },
      });

      if (!response.ok) throw new Error('Failed to fetch news');

      const articles = await response.json();
      const data: NewsItem[] = articles.data;

      cache[cacheKey] = {
        timestamp: Date.now(),
        data
      };

      setNews(data);
      setLastUpdated(new Date());
    } catch (err) {
      setError(err instanceof Error ? err.message : 'An error occurred');
    } finally {
      setLoading(false);
    }
  };

  // Fetch data when component mounts
  useEffect(() => {
    fetchNews(TOPICS);
  }, []);

  // Handler for manual refresh
  const handleRefresh = () => {
    const cacheKey = TOPICS.sort().join(',');
    delete cache[cacheKey];
    fetchNews(TOPICS);
  };

  return (
    <div className={styles.container}>
      <PriceTicker />

      {lastUpdated && (
        <div className={styles.lastUpdated}>
          Last updated: {lastUpdated.toLocaleTimeString()}
          <button onClick={handleRefresh} className={styles.refreshButton}>
            Refresh
          </button>
        </div>
      )}

      {error && <div className={styles.error}>{error}</div>}

      {loading ? (
        <div className={styles.loading}>Loading...</div>
      ) : (
        news.map((item: NewsItem, index: number) => (
          <NewsCard key={item.id || index} item={item} index={index} />
        ))
      )}
    </div>
  );
}

Update page.module.css with the following styles:

.container {
  max-width: 800px;
  margin: 0 auto;
  padding: 20px;
}

.topics {
  margin-bottom: 20px;
}

.topic {
  margin-right: 10px;
  padding: 5px 10px;
  border: 1px solid #ddd;
  border-radius: 4px;
  background: none;
  cursor: pointer;
}

.topicSelected {
  background: #007bff;
  color: white;
  border-color: #007bff;
}

.error {
  color: #dc3545;
  padding: 10px;
  border: 1px solid #dc3545;
  border-radius: 4px;
  margin-bottom: 15px;
}

.loading {
  text-align: center;
  padding: 20px;
}

.lastUpdated {
  color: #666;
  font-size: 0.9em;
  margin-bottom: 15px;
}

.refreshButton {
  background: #007bff;
  color: white;
  border: none;
  padding: 5px 10px;
  border-radius: 4px;
  cursor: pointer;
  margin-left: 10px;
}

Update app/layout.tsx with the following:

export default function RootLayout({
  children,
}: {
  children: React.ReactNode
}) {
  return (
    <html lang="en">
      <head>
        <title>Crypto News Aggregator Application</title>
        <meta name="description" content="Crypto News Aggregator Application" />
      </head>
      <body>
        {children}
      </body>
    </html>
  )
}

Running your project

Go ahead and run your project

npm run dev

You can find your running app at https://localhost:3000

Congrats on finishing the project! 🏆 👏

Testing the Page Component

We are going to add a test for the Home component that verifies that it correctly renders news content after fetching data. Go ahead and create this test file.

__tests__/Home.test.tsx


import { render, screen, waitFor } from '@testing-library/react';
import Home from '@/app/page';

describe('Home', () => {
  beforeEach(() => {
    // Set up test environment variables
    process.env.NEXT_PUBLIC_API_URL = 'http://test-api.com';
    process.env.NEXT_PUBLIC_API_TOKEN = 'test-token';

    // Mock fetch for both API endpoints
    global.fetch = jest.fn((url) => {
      // Mock responses for different API calls
      if (url.includes('api.coingecko.com')) {
        return Promise.resolve({
          ok: true,
          json: () => Promise.resolve({
            bitcoin: { usd: 65000, usd_24h_change: 2.5 }
          }),
          status: 200,
        } as Response);
      }
      return Promise.resolve({
        ok: true,
        json: () => Promise.resolve({
          data: [{
            id: '1',
            title: 'News Title',
            description: 'News Description',
            url: 'https://test.com',
            published_at: '2024-03-25'
          }]
        }),
        status: 200,
      } as Response);
    }) as jest.Mock;
  });

  test('renders news feed', async () => {
    render(<Home />);
    await waitFor(
      () => expect(screen.getByText("News Title")).toBeInTheDocument(),
      { timeout: 3000 }
    );
  });
});

Run the test

npm run test

This test verifies that our Home component successfully renders news content after fetching data.

Consider improving this project further.

Some ideas:

Implement proper loading states
Add pagination for news items
Implement more sophisticated caching
Enhance the testing suite
You can change the topic query param to fetch different types of news

Thanks for following along! 😄

Cover image credit: Photo by RDNE Stock project: https://www.pexels.com/photo/selective-focus-photo-of-silver-and-gold-bitcoins-8369648/

DEV Community: NewsDataHub

Understanding Pagination with NewsDataHub API

Paginating through results with Python

Index based Pagination

Crypto News Aggregator using Typescript, Next.js, NewsDataHub and CoinGecko APIs

Project Structure Setup

The files that can be safely deleted

Testing

Getting Your NewsDataHub API Token

Adding the API key to Your Project

Replace your_token_here in .env.local with your NewsDataHub API token from your dashboard.

Configuration Files Setup and Overview

Updating .gitignore

Setting up Environment Type Definitions

ESLint Setup

Now let’s add the project code

NewsCard and PriceTicker Component Implementation

NewsCard component

PriceTicker Component

Building the Main Page Component

Running your project

Testing the Page Component

Consider improving this project further.

Thanks for following along! 😄

Crypto News Aggregator using Typescript, Next.js, NewsDataHub and CoinGecko APIs

Project Structure Setup

The files that can be safely deleted

Testing

Getting Your NewsDataHub API Token

Adding the API key to Your Project

Replace your_token_here in .env.local with your NewsDataHub API token from your dashboard.

Configuration Files Setup and Overview

Updating .gitignore

Setting up Environment Type Definitions

ESLint Setup

Now let’s add the project code

NewsCard and PriceTicker Component Implementation

NewsCard component

PriceTicker Component

Building the Main Page Component

Running your project

Testing the Page Component

Consider improving this project further.

Thanks for following along! 😄

Replace `your_token_here` in `.env.local` with your NewsDataHub API token from your dashboard.

Replace `your_token_here` in `.env.local` with your NewsDataHub API token from your dashboard.