DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Rocktim Saikia
Rocktim Saikia

Posted on

meta-fetcher: URL meta-data fetcher for Node.js

Hi, Myself Rocktim Saikia. And today I want to showcase a simple project that I have been working on for couple of days now.

meta-fetcher πŸ”Ž

meta-fecther is simple and tiny url meta-data fetcher for Nodejs. Under the hood this module uses node-fetch to fetch the meta tags of given website url and returns the parsed meta-data as a JSON object.

GitHub logo rocktimsaikia / meta-fetcher

simple metadata scrapper for node.js

meta-fetcher

GitHub Workflow Status (branch) npm

Simple metadata scrapper for node.js. Under the hood it uses isomorphic-unfetch
to fetch the metadata, parses it and returns it as json object.


Installation

npm install meta-fetcher
Enter fullscreen mode Exit fullscreen mode

or use yarn

yarn add meta-fetcher
Enter fullscreen mode Exit fullscreen mode

Usage

import metaFetcher from 'meta-fetcher';

(async () => {
  const result = await metaFetcher('https://hoppscotch.io/');
  console.log(result);

  /*
        {
            metadata: {
                website: 'https://hoppscotch.io/',
                title: 'Hoppscotch - Open source API development ecosystem',
                description: 'Helps you create requests faster, saving precious time on development.',
                banner: 'https://hoppscotch.io/banner.jpg',
                themeColor: '#202124'
            },
            socials: {
                'twitter:site': '@hoppscotch_io',
                'twitter:creator': '@hoppscotch_io'
            },
            favicons: [
                'https://hoppscotch.io/_nuxt/icons/icon_64x64.9834b3.png',
                'https://hoppscotch.io/_nuxt/icons/icon_512x512.9834b3.png'
            ]
        }
    */
})();
Enter fullscreen mode Exit fullscreen mode

API

metaFetcher(input)

Takes one url string as a parameter and returns an object containing the meta-information.

input

type: string
default: 'none'

The url string to be scrapped.


Related

Here are some related projects

Install

npm install meta-fetcher
Enter fullscreen mode Exit fullscreen mode

Basic Usage

const {fetchMetaData} = require('meta-fetcher');

(async () => {
    const result = await fetchMetaData('https://hoppscotch.io/');
        console.log(result);

    /*
        {
        basic_metadata: {
            website: 'https://hoppscotch.io/',
            title: 'Hoppscotch β€’ A free, fast and beautiful API request builder',
            description: 'A free, fast and beautiful API request builder'
        },
        opengraph: {
            'og:image': 'https://hoppscotch.io/banner.jpg',
            'og:type': 'website',
            'og:title': 'Hoppscotch',
            'og:site_name': 'Hoppscotch',
            'og:description': 'A free, fast and beautiful API request builder',
            'og:url': 'https://hoppscotch.io/'
        },
        opengraph_social: {
            'twitter:card': 'summary_large_image',
            'twitter:site': '@liyasthomas',
            'twitter:creator': '@liyasthomas'
        },
        favicons: [
            'https://hoppscotch.io/icon.png',
            'https://hoppscotch.io/icon.png',
            'https://hoppscotch.io/_nuxt/icons/icon_64x64.9834b3.png'
        ]
        }
    */ 
})();

Enter fullscreen mode Exit fullscreen mode

It can also fetch meta-data from shortened-url.
For example:

const {fetchMetaData} = require('meta-fetcher');

(async () => {
    const result = await fetchMetaData('https://bit.ly/2Fj9sNF');
    console.log(result);
})();
Enter fullscreen mode Exit fullscreen mode

There a few advanced options that you can use before fetching. You can check them in the readme.md section.

If you like the project. Do leave a star at repo 🌟

Top comments (4)

Collapse
 
ramanjaneyakarnati profile image
Ramanjaneya K

Best metadata fetching tools i came across: bit.ly/3sQHiwg

Collapse
 
shoaibsharif profile image
Shoaib

I really like the idea of your website. I have noticed that you used netlify function. Would you mind sharing us what did you use to fetch metadata on your website?

Collapse
 
yellow1912 profile image
yellow1912

Thanks for sharing. Starred. I may need it later.

Collapse
 
rocktimsaikia profile image
Rocktim Saikia Author

Sure 😊 . Glad you liked it

🌚 Browsing with dark mode makes you a better developer by a factor of exactly 40.

It's a scientific fact.