DEV Community

bao001 xiao
bao001 xiao

Posted on

Build Your Own "Read Later" App in 50 Lines of Python with the Web to Markdown/JSON API

Have you ever found yourself drowning in browser tabs of articles you swear you will read "later"? I have been there. So I built a dead-simple reading list app that strips web pages down to clean, distraction-free Markdown — and it only took 50 lines of Python.

All thanks to the Web to Markdown/JSON API, a free-tier friendly service that converts any URL into clean, structured content.

What We Are Building

A CLI reading list tool that:

  1. Accepts a URL from the command line
  2. Fetches and converts the page to Markdown via the API
  3. Saves it locally with metadata (title, author, date saved)
  4. Lets you list and search your saved articles

Here is what it looks like:

# Save an article
python readlater.py add https://blog.rust-lang.org/2024/02/21/Rust-1.76.0.html

# List your reading list
python readlater.py list

# Search by keyword
python readlater.py search "async"
Enter fullscreen mode Exit fullscreen mode

The Magic Behind It: The Web to Markdown/JSON API

Before we dive into code, let me show you the API that makes this possible. It is dead simple:

curl -X POST https://web2md-api-production-d822.up.railway.app/extract \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "format": "markdown"}'
Enter fullscreen mode Exit fullscreen mode

You get back clean, structured Markdown:

{
  "success": true,
  "title": "Example Domain",
  "content": "# Example Domain\n\nThis domain is for use...",
  "description": "",
  "author": "",
  "published_date": "",
  "word_count": 24,
  "response_time_ms": 177
}
Enter fullscreen mode Exit fullscreen mode

Three formats are supported: markdown, json (structured JSON representation), and text (plain text). You can also cap the response with max_length to keep things snappy.

The best part? The free tier gives you 50 requests per day — more than enough for personal use. If you need more, paid plans are available on RapidAPI.

Building the Reading List App

Let us write the code. Create a file called readlater.py:

#!/usr/bin/env python3
"""A simple read-later app powered by the Web to Markdown/JSON API."""

import json
import os
import sys
import sqlite3
from datetime import datetime
from pathlib import Path

import requests

# --- Configuration ---
API_URL = "https://web2md-api-production-d822.up.railway.app/extract"
DB_PATH = Path.home() / ".readlater" / "library.db"
ARTICLES_DIR = Path.home() / ".readlater" / "articles"


# --- Database Setup ---
def init_db():
    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
    ARTICLES_DIR.mkdir(parents=True, exist_ok=True)
    conn = sqlite3.connect(DB_PATH)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS articles (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            url TEXT UNIQUE NOT NULL,
            title TEXT,
            author TEXT,
            description TEXT,
            published_date TEXT,
            word_count INTEGER,
            saved_at TEXT NOT NULL,
            filename TEXT NOT NULL
        )
    """)
    conn.commit()
    return conn


# --- Fetch Article via API ---
def fetch_article(url: str) -> dict:
    """Convert a URL to Markdown using the Web to Markdown/JSON API."""
    payload = {"url": url, "format": "markdown", "max_length": 50000}
    response = requests.post(API_URL, json=payload, timeout=30)
    response.raise_for_status()
    data = response.json()
    if not data.get("success"):
        raise RuntimeError(f"API error: {data.get('error', 'Unknown')}")
    return data


# --- Save Article ---
def save_article(url: str) -> None:
    """Fetch and save an article to the local library."""
    print(f"Fetching: {url}")
    data = fetch_article(url)

    conn = init_db()
    saved_at = datetime.now().isoformat()
    safe_title = "".join(c for c in data["title"] if c.isalnum() or c in " _-").rstrip()
    filename = f"{saved_at[:10]}_{safe_title[:50]}.md"
    filepath = ARTICLES_DIR / filename

    # Write Markdown with metadata header
    with open(filepath, "w", encoding="utf-8") as f:
        f.write(f"---\n")
        f.write(f"title: {data['title']}\n")
        f.write(f"url: {url}\n")
        f.write(f"author: {data.get('author', 'N/A')}\n")
        f.write(f"saved_at: {saved_at}\n")
        f.write(f"word_count: {data.get('word_count', 0)}\n")
        f.write(f"---\n\n")
        f.write(data["content"])

    conn.execute(
        "INSERT OR REPLACE INTO articles(url, title, author, description, published_date, word_count, saved_at, filename) VALUES(?, ?, ?, ?, ?, ?, ?, ?)",
        (url, data["title"], data.get("author"), data.get("description"),
         data.get("published_date"), data.get("word_count"), saved_at, filename),
    )
    conn.commit()
    conn.close()

    print(f"Saved: {data['title']}")
    print(f"   Words: {data.get('word_count', 'N/A')} | File: {filename}")


# --- List and Search ---
def list_articles(search_term: str = None) -> None:
    """Display saved articles, optionally filtered by search term."""
    conn = init_db()
    if search_term:
        rows = conn.execute(
            "SELECT title, author, word_count, saved_at, url FROM articles "
            "WHERE title LIKE ? OR description LIKE ? ORDER BY saved_at DESC",
            (f"%{search_term}%", f"%{search_term}%"),
        ).fetchall()
        print(f'Results for "{search_term}":\n')
    else:
        rows = conn.execute(
            "SELECT title, author, word_count, saved_at, url FROM articles ORDER BY saved_at DESC"
        ).fetchall()
        print(f"Your Reading List ({len(rows)} articles):\n")

    for i, (title, author, wc, saved, url) in enumerate(rows, 1):
        print(f"{i}. {title}")
        print(f"   {wc} words | {saved[:10]} | {url[:60]}...")
        print()
    conn.close()


# --- CLI ---
if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python readlater.py [add <url> | list | search <term>]")
        sys.exit(1)

    command = sys.argv[1]
    if command == "add" and len(sys.argv) == 3:
        save_article(sys.argv[2])
    elif command == "list":
        list_articles()
    elif command == "search" and len(sys.argv) == 3:
        list_articles(sys.argv[2])
    else:
        print("Usage: python readlater.py [add <url> | list | search <term>]")
Enter fullscreen mode Exit fullscreen mode

How It Works

Let us walk through the key parts:

1. The API Call

The fetch_article() function sends a POST request to the API endpoint. We ask for markdown format and cap at 50,000 characters — plenty for long-form articles. The API returns rich metadata (title, author, word count) alongside the cleaned content.

2. Local Storage

Each article is saved as a .md file with YAML frontmatter, so it is readable by any Markdown viewer — even Obsidian or VS Code. Metadata is also stored in a SQLite database for fast searching.

3. Search

A simple SQL LIKE query lets you find articles by title or description. No fancy vector embeddings needed for a personal library.

Taking It Further

This is just the foundation. Here are a few ways to extend it:

  • Export to Kindle: Convert the Markdown to an EPUB and email it to your device (pandoc makes this trivial)
  • Weekly digest email: Cron job that picks 3 random unread articles and emails them to you
  • Browser extension: Right-click any page and send it straight to your reading list
  • AI summaries: Pipe the content through an LLM to get TL;DRs for each saved article
  • RSS integration: Combine with an RSS feed parser to auto-save articles from your favorite blogs

The API also returns json format if you prefer structured data over Markdown:

# Get structured JSON instead of Markdown
payload = {"url": "https://example.com", "format": "json"}
response = requests.post(API_URL, json=payload)
data = response.json()
# data["content"] is now a structured JSON tree, not a string
Enter fullscreen mode Exit fullscreen mode

Why This API?

I have tried several web-to-markdown services and scrapers. Here is what sets this one apart:

Feature This API BeautifulSoup DIY Readability clones
No code maintenance Yes No No
Handles JS-rendered pages Yes No No
Consistent output format Yes Depends on site Varies
Free tier available Yes (50/day) N/A Often none
Response time ~200ms Self-hosted Varies

It is fast, reliable, and I do not have to maintain a headless browser or update CSS selectors when websites redesign.

Get Started

  1. Try the API: Hit the endpoint directly with curl or your favorite HTTP client
  2. RapidAPI subscribers: Get higher rate limits at RapidAPI
  3. Clone and extend: The full reading list code above is yours to use and modify
# Quick test — extract any URL!
curl -X POST https://web2md-api-production-d822.up.railway.app/extract \
  -H "Content-Type: application/json" \
  -d '{"url": "https://dev.to", "format": "markdown", "max_length": 5000}'
Enter fullscreen mode Exit fullscreen mode

Stop letting articles pile up in your tabs. Build your reading list today!


Have questions or built something cool with the API? Drop a comment below!

Top comments (0)