DEV Community

clause-netizen
clause-netizen

Posted on • Originally published at github.com

Build a link-preview service with one API call

#webdev #javascript #tutorial #api

When someone pastes a URL into your app's comment box or chat, you want to show a preview card: the page title, a short description, the favicon, and the social image. Doing that yourself means fetching the HTML, parsing the <head>, reconciling OpenGraph tags with Twitter cards with bare <meta> tags, chasing redirects, and handling the sites that return garbage. It's a weekend of work and a permanent maintenance tax.

SiteIntel does that parsing for you and hands back clean JSON. One GET request per URL.

The request

The API lives at https://siteintel.p.rapidapi.com and uses RapidAPI's auth headers. Here's a preview for a GitHub repo:

curl --request GET \
  --url 'https://siteintel.p.rapidapi.com/analyze?url=https://github.com/clause-netizen/siteintel-api' \
  --header 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  --header 'X-RapidAPI-Host: siteintel.p.rapidapi.com'
Enter fullscreen mode Exit fullscreen mode

The same thing in Node, using the built-in fetch (Node 18+, no dependencies):

async function getPreview(targetUrl) {
  const endpoint = new URL('https://siteintel.p.rapidapi.com/analyze');
  endpoint.searchParams.set('url', targetUrl);

  const res = await fetch(endpoint, {
    headers: {
      'X-RapidAPI-Key': process.env.RAPIDAPI_KEY,
      'X-RapidAPI-Host': 'siteintel.p.rapidapi.com',
    },
  });

  if (!res.ok) {
    throw new Error(`SiteIntel returned ${res.status}`);
  }
  return res.json();
}

const preview = await getPreview('https://stripe.com');
console.log(preview.title, '—', preview.description);
Enter fullscreen mode Exit fullscreen mode

What you get back

The response gives you the fields a preview card needs, already normalized:

{
  "url": "https://stripe.com",
  "title": "Stripe | Financial Infrastructure to Grow Your Revenue",
  "description": "Stripe powers online and in-person payment processing...",
  "favicon": "https://stripe.com/favicon.ico",
  "ogImage": "https://stripe.com/img/v3/home/social.png"
}
Enter fullscreen mode Exit fullscreen mode

Two things worth knowing from having run this in production. The favicon and ogImage come back as absolute URLs, so you don't have to resolve relative paths against the origin yourself. And description falls back across sources: it prefers the OpenGraph og:description, then the standard meta description, so you usually get something rather than null even on pages that skip OG tags.

One thing to build with it

A comment system that turns pasted links into cards. When a user submits a comment, scan the text for URLs, call the API for each one, and store the result alongside the comment:

async function enrichComment(text) {
  const urls = text.match(/https?:\/\/[^\s]+/g) || [];
  const previews = await Promise.all(
    urls.map((u) => getPreview(u).catch(() => null))
  );
  return { text, previews: previews.filter(Boolean) };
}
Enter fullscreen mode Exit fullscreen mode

The .catch(() => null) matters. Some links will 404 or time out, and you don't want one dead URL to sink the whole comment. Cache previews by URL too, since the same links get pasted over and over and the metadata rarely changes within a day.

That's the whole thing. Fetch, render the card, cache the result.

Working examples and the full field list are in the repo: https://github.com/clause-netizen/siteintel-api. If you'd rather not manage your own scraping infra, it's on RapidAPI with a managed key: https://rapidapi.com/hidanny0001/api/siteintel

Top comments (0)