meta-fetcher: URL meta-data fetcher for Node.js

Hi, Myself Rocktim Saikia. And today I want to showcase a simple project that I have been working on for couple of days now.

meta-fetcher 🔎

meta-fecther is simple and tiny url meta-data fetcher for Nodejs. Under the hood this module uses node-fetch to fetch the meta tags of given website url and returns the parsed meta-data as a JSON object.

rocktimsaikia / meta-fetcher

simple metadata scrapper for node.js

meta-fetcher

Simple metadata scrapper for node.js. Under the hood it uses isomorphic-unfetch
to fetch the metadata, parses it and returns it as json object.

Installation

npm install meta-fetcher

or use yarn

yarn add meta-fetcher

Usage

import metaFetcher from 'meta-fetcher';

(async () => {
  const result = await metaFetcher('https://hoppscotch.io/');
  console.log(result);

  /*
        {
            metadata: {
                website: 'https://hoppscotch.io/',
                title: 'Hoppscotch - Open source API development ecosystem',
                description: 'Helps you create requests faster, saving precious time on development.',
                banner: 'https://hoppscotch.io/banner.jpg',
                themeColor: '#202124'
            },
            socials: {
                'twitter:site': '@hoppscotch_io',
                'twitter:creator': '@hoppscotch_io'
            },
            favicons: [
                'https://hoppscotch.io/_nuxt/icons/icon_64x64.9834b3.png',
                'https://hoppscotch.io/_nuxt/icons/icon_512x512.9834b3.png'
            ]
        }
    */
})();

API

metaFetcher(input)

Takes one url string as a parameter and returns an object containing the meta-information.

input

type: string
default: 'none'

The url string to be scrapped.

Related

Here are some related projects

• page-scrapper: 📦 node.js scrapper that…

View on GitHub

Install

npm install meta-fetcher

Basic Usage

const {fetchMetaData} = require('meta-fetcher');

(async () => {
    const result = await fetchMetaData('https://hoppscotch.io/');
        console.log(result);

    /*
        {
        basic_metadata: {
            website: 'https://hoppscotch.io/',
            title: 'Hoppscotch • A free, fast and beautiful API request builder',
            description: 'A free, fast and beautiful API request builder'
        },
        opengraph: {
            'og:image': 'https://hoppscotch.io/banner.jpg',
            'og:type': 'website',
            'og:title': 'Hoppscotch',
            'og:site_name': 'Hoppscotch',
            'og:description': 'A free, fast and beautiful API request builder',
            'og:url': 'https://hoppscotch.io/'
        },
        opengraph_social: {
            'twitter:card': 'summary_large_image',
            'twitter:site': '@liyasthomas',
            'twitter:creator': '@liyasthomas'
        },
        favicons: [
            'https://hoppscotch.io/icon.png',
            'https://hoppscotch.io/icon.png',
            'https://hoppscotch.io/_nuxt/icons/icon_64x64.9834b3.png'
        ]
        }
    */ 
})();

It can also fetch meta-data from shortened-url.
For example:

const {fetchMetaData} = require('meta-fetcher');

(async () => {
    const result = await fetchMetaData('https://bit.ly/2Fj9sNF');
    console.log(result);
})();

There a few advanced options that you can use before fetching. You can check them in the readme.md section.

If you like the project. Do leave a star at repo 🌟

Comments

[yellow1912]

Thanks for sharing. Starred. I may need it later.

[Rocktim Saikia]

Sure 😊 . Glad you liked it

[Ramanjaneya K]

Best metadata fetching tools i came across: bit.ly/3sQHiwg

[Shoaib]

I really like the idea of your website. I have noticed that you used netlify function. Would you mind sharing us what did you use to fetch metadata on your website?