How to make a CLI based movie scraper using NodeJS

Dave Shilander — Mon, 02 May 2022 02:03:18 +0000

How to make a CLI based movie scraper using NodeJS

This guide will instruct you on how to make a CLI application that will ask the user for movie, and present info about that movie in return. To be most successful you should have at least a beginers understanding of Node and NPM. You should also have a basic understanding of using bash to navigate files. Although you are free to use a GUI application if you wish, only terminal examples will be provided.

Prerequsites

A MovieDB API Key

You will need a free acount with TheMovieDB to obtain and API key. You can sign up for one here
Once you have a key, you can consult the API Documentation
Node JS

You will need to have Node JS installed. Instruction for various platforms can be found here
A text editor or IDE of your choice.
A terminal that is capable of running bash commands.

Part One (Setup Folder & File Structure, Install NPM Packages)

First, create the root folder for the project. This guide will call it movie-scraper. Then navigate into the newly created folder.

mkdir movie-scraper && cd movie-scraper

Within the movie-scraper directory run the following command.

npm init -y

This will create the package.json file. You need to make two modifications to this file.
Add "type": "module" to the top level, and "start": "node src/index.js" to the scripts section.

It should look like this. (You may have some slight differences depending on your pesoral npm settings, just focus on the changes needed above)

{
  "name": "movie-scraper",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "start": "node src/index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
  }
}

Run this next command in your terminal to install the npm packages axios dotenv and inquirer

npm i axios dotenv inquirer

You should now see a node-modules directory in your root folder.

.
├── node_modules
├── package-lock.json
├── package.json

You should also see the newly installed packages listed in your package.json dependcies

"dependencies": {
    "axios": "^0.26.1",
    "dotenv": "^16.0.0",
    "inquirer": "^8.2.2"
}

Now you are going to create a .env file that will hold your API key for TheMovieDB. If you are using git or any other version control for this project, remember to not commit this file.

Replace PUT_YOUR_API_KEY_HERE with your actual API key. Keep the backslashes and quotation marks.

echo TMDB_API_KEY=\"PUT_YOUR_API_KEY_HERE\" >> .env

You can verify the file was created using the cat command

cat .env

You will see th following output with your API key in place of the Xs

TMDB_API_KEY="XXXXXXXXXX"

Create the src directory that will contain the logic for the program, and navigate into it.

mkdir src && cd src

Create the the following files in the src directory:

Movie.js
config.js
getByID.js
index.js
prompt.js
searchMovie.js

You can do this in one command in your terminal

touch {Movie,config,getByID,index,prompt,searchMovie}.js

Your project should now have the following file stucture

.
├── .env
├── node_modules
├── package-lock.json
├── package.json
└── src
    ├── Movie.js
    ├── config.js
    ├── getByID.js
    ├── index.js
    ├── prompt.js
    └── searchMovie.js

Part Two (Javascript Files)

Copy/Paste the following code to each correspodning file

`src/config.js`

import dotenv from "dotenv";
dotenv.config();

export default{
    tmbdkey: process.env.TMDB_API_KEY,
};

`src/getByID.js`

import config from "./config.js";
import axios from 'axios'

const getByID = async (id) => {
        const options={
            params: {
                api_key: config.tmbdkey,
                language: "en-US",
                append_to_response: `credits`
            }
    }

    let response = await axios.get(`https://api.themoviedb.org/3/movie/${id}`, options)
    return response.data

}

export default getByID;

`src/index.js`

import inquirer from 'inquirer';
import Movie from './Movie.js'
import moviePrompts from './prompt.js'

const movieResponse = await inquirer.prompt(moviePrompts)
const selectedMovie = await new Movie(movieResponse.movieID);
selectedMovie.summary()

`src/Movie.js`

import getByID from './getByID.js'

class Movie {
    constructor(id) {
        return (async () => {
            this.details = await getByID(id)
            return this
        })()
    }

    get title() {
        return this.details.title
    }

    get tagline() {
        return this.details.tagline
    }

    get overview() {
        return this.details.overview
    }

    get directors() {
        const directors = this.details.credits.crew.filter(
            (obj) => obj.job === 'Director',
        )
        return directors.map((director) => director.name)
    }

    get writers() {
        const writers = this.details.credits.crew.filter(
            (obj) => obj.job === 'Screenplay',
        )
        return writers.map((writer) => writer.name)
    }

    get cast() {
        const cast = this.details.credits.cast.slice(0, 5)
        return cast.map((castMember) => ({
            name: castMember.name,
            role: castMember.character,
        }))
    }

    summary() {
        const summary = 
`
${this.title} - ${this.tagline}
Directed By: ${this.directors}  Written By: ${this.writers}

${this.overview}

Starring:
${this.cast.map(castMember => ` ${castMember.name} as ${castMember.role}`)}
`
        console.log(summary) 
    }
}

export default Movie

`src/prompt.js`

import searchMovie from './searchMovie.js'

const moviePrompts = [  
    {
        name: "name",
        type: "input",
        message: "Enter a movie to search:"
    },
    {
        name: "movieID",
        type: "list",
        message: "Select a movie:",
        choices: (answers) => searchMovie(answers.name)
    }
]

export default moviePrompts

`src/searchMovie.js`

import config from "./config.js";
import axios from 'axios'

const searchMovie = async (movie) => {
        const options={
            params: {
                api_key: config.tmbdkey,
                language: "en-US",
                query: movie,
            }
    }

    const simpleList = movieObj => ({name: `${movieObj.title} (${movieObj.release_date.slice(0,4)})`, value: movieObj.id})

    const res = await axios.get('https://api.themoviedb.org/3/search/movie', options);

    const list = res.data.results.map(simpleList).slice(0,5)

    return list;

}

export default searchMovie;

Part 3 (Execution)

At the root of your project run

npm start

You will be preseted with the following.

> movie-scraper@1.0.0 start
> node src/index.js

? Enter a movie to search:

Follow the prompts as given by the application.

DEV Community: Dave Shilander

How to make a CLI based movie scraper using NodeJS

How to make a CLI based movie scraper using NodeJS

Prerequsites

Part One (Setup Folder & File Structure, Install NPM Packages)

Part Two (Javascript Files)

src/config.js

src/getByID.js

src/index.js

src/Movie.js

src/prompt.js

src/searchMovie.js

Part 3 (Execution)

`src/config.js`

`src/getByID.js`

`src/index.js`

`src/Movie.js`

`src/prompt.js`

`src/searchMovie.js`