How to Extract Metadata from Any URL in 5 Lines of Code (Free API Tutorial)

#python #api #webdev #tutorial

How to Extract Metadata from Any URL in 5 Lines of Code

Every time you paste a link into Slack, Discord, or Twitter, the platform fetches that page's metadata — title, description, image, author — and renders a rich preview. That's Open Graph and Twitter Cards at work.

Here's how to do the same thing in your own app, using the {{metadata-extractor}} on Apify.

What You Get

For any URL, the actor returns Open Graph tags, Twitter Cards, JSON-LD structured data, standard meta tags, and technical details like favicon and canonical URL. All in one JSON response. No browser needed — the actor uses aiohttp for sub-second extraction.

Quick Start

import requests

API_URL = "https://api.apify.com/v2/acts/weeknds~link-preview-metadata-extractor/run-sync-get-dataset-items"
API_TOKEN = "YOUR_APIFY_TOKEN"

response = requests.post(
    f"https://api.apify.com/v2/acts/weeknds~link-preview-metadata-extractor/runs",
    headers={"Authorization": f"Bearer {API_TOKEN}"},
    json={"url": "https://example.com"}
)

run_id = response.json()["data"]["id"]

results = requests.get(
    f"https://api.apify.com/v2/acts/weeknds~link-preview-metadata-extractor/runs/{run_id}/dataset/items",
    headers={"Authorization": f"Bearer {API_TOKEN}"}
)

metadata = results.json()[0]
print(f"Title: {metadata['title']}")
print(f"Description: {metadata['description']}")
print(f"Image: {metadata.get('og:image', 'No image')}")

Or with curl:

curl -X POST "https://api.apify.com/v2/acts/weeknds~link-preview-metadata-extractor/runs" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

curl "https://api.apify.com/v2/acts/weeknds~link-preview-metadata-extractor/runs/RUN_ID/dataset/items" \
  -H "Authorization: Bearer YOUR_TOKEN"

Building a Link Preview Card

def build_preview_card(url):
    import requests, time

    resp = requests.post(
        "https://api.apify.com/v2/acts/weeknds~link-preview-metadata-extractor/runs",
        headers={"Authorization": f"Bearer {API_TOKEN}"},
        json={"url": url}
    ).json()

    run_id = resp["data"]["id"]
    time.sleep(3)

    items = requests.get(
        f"https://api.apify.com/v2/acts/weeknds~link-preview-metadata-extractor/runs/{run_id}/dataset/items",
        headers={"Authorization": f"Bearer {API_TOKEN}"}
    ).json()

    meta = items[0] if items else {}

    return f"""
    <div class="link-preview">
        <img src="{meta.get('og:image', '')}"
             alt="{meta.get('og:title', meta.get('title', ''))}"
             onerror="this.style.display='none'" />
        <div class="preview-text">
            <h4>{meta.get('og:title', meta.get('title', 'Unknown'))}</h4>
            <p>{meta.get('og:description', meta.get('description', ''))}</p>
            <small>{meta.get('og:site_name', '')}</small>
        </div>
    </div>
    """

Real-World Uses

Slack Bot Link Unfurling

Connect to Slack's Events API, listen for link_shared events, call the metadata extractor, and post an unfurl. Works exactly like native Slack link previews.

SEO Audit Tool

Batch-check metadata across your site to find missing Open Graph tags. Every page without og:title and og:description is losing clicks on social media.

Content Aggregator

Pull structured data from blog posts, news articles, and product pages. The JSON-LD extraction gives you schema.org markup — articles have author and publish date, products have price and availability, events have location and time.

Why Not Build It Yourself

You could write your own metadata extractor. Fetch the HTML, parse meta tags, handle redirects. Not hard. But then you deal with encoding detection (sites that lie about their charset), redirect chains (short URLs, tracking links), JavaScript-rendered meta tags, and nested JSON-LD parsing.

The actor handles all of this. One endpoint, clean JSON back.

At $0.002 per run, you can extract metadata from 500 URLs for a dollar. For most use cases — link unfurling, SEO audits, content aggregation — that's way cheaper than running your own extraction infrastructure. If you need metadata from a specific type of page the actor doesn't handle yet, let me know. I actively maintain it.

DEV Community