DEV Community: Neeraj Sewani

Mock Screenshot feature of SaaS applications

Neeraj Sewani — Tue, 02 Nov 2021 04:26:18 +0000

Well, while working on a SaaS application for a client recently, I came across a feature where the application allows you to take the screenshot of the entire page and have it sent in the email to the requested user either in the Image or PDF format. I got intrigued by this feature and thought of sharing it here.

STEP-1: The setup - We are going to need a backend which I have designed using Node.js and Express.js to take the screenshot and mail it along with the frontend. The Backend and Frontend can be found in the repos attached.

STEP-2: The UI - Honestly, the UI of a SasS application can be quite boring as it just displays a lot of boring data consists of metric cards, goal scorecards, charts etc. so to make this blog a little interesting the UI displays a collection of Pokemon cards, because who doesn't like Pokemons, eh?

STEP-3: The backend - Now since the UI is there, let's build the backend. It is gonna be quite simple. It just gonna have a route which would be called when the "Export Screenshot" button from the frontend is clicked.

const express = require("express")
let cors = require("cors")

const { sendEmail } = require("./helpers/helper")

const app = express()
app.use(cors())
app.use(express.json())

app.get("/", (req, res) => {
  res.send("backend home route")
})

app.post("/send-screenshot", async (req, res) => {
  const { receiversEmail } = req.body

  try {
    await sendEmail(receiversEmail)
    res.status(200).send("Email sent successfully")
  } catch (err) {
    res.status(400).send("Error in sending the email with screenshot")
  }
})

app.listen(4000, () => {
  console.info("Backend is running on port 4000")
})

STEP-4: The Screenshot - Now, since the frontend and backend are in place, let's use the Screenshot API's query builder to design a query for the screenshot.

Here, I have designed a query in order to get a high resolution, full page screenshot of the current page.

Following are the options that I am using explained:

Full page screenshot - it means the entire page including the scrollable area.
Retina - This would make the screenshot gets captured in high resolution
Lazy loading - this will make sure that all the content is loaded before the screenshot is taken
Fresh screenshot - The screenshot.api caches all the screenshots for you on their server so, using this just to make sure that I get a new screenshot every time. Alternatively, you can use "Destroy screenshot" feature which makes sure that your screenshot is not cached on their server in case the data displayed is sensitive.
Load event - A lot of nuances lies with this feature and using it correctly would save a lot of time. By default it is set to "load" but imagine a scenario where the page has a lot of images and obviously those images would take some time to load up, now since you want to make sure that all the images on the page show up in the screenshot therefore, we need to use the option "networkidle" here which essentially means that the API is going to wait until all the network calls are completed and then only it is going to take the screenshot.

Moreover, if you want to take screenshot of any public website that then following two options are going to be of utmost importance. "Block ads" and "no Cookie banners".

Finally, the query would look something like this
https://shot.screenshotapi.net/screenshot?token=<YOUR_API_TOKEN>&url=<FORNTEND_URL>&full_page=true&fresh=true&output=image&file_type=jpeg&lazy_load=true&retina=true&wait_for_event=networkidle

PS. For the frontend URL ngrok can be used.

STEP-5: The Email - We are gonna use nodemailer for sending the screenshot. The screenshot.api would send back the JSON response which would contain the screenshot key that would contain the URL of the screenshot. Now, for emailing the image we need to first fetch the image, write it to the disk using the fs module and then send it using nodemailer. Attaching the code to the below:

const nodemailer = require("nodemailer")
const axios = require("axios")
const fs = require("fs")

const { SCREENSHOT_API_TOKEN } = require("./credentials")
const path = require("path")

const takeScreenshot = async () => {
  try {
    var query = "https://shot.screenshotapi.net/screenshot"
    let url = "<FRONTEND_URL>"
    query += `?token=${SCREENSHOT_API_TOKEN}&url=${url}&full_page=true&fresh=true&output=image&file_type=jpeg&lazy_load=true&retina=true&wait_for_event=networkidle`
    const response = await axios.get(query)

    console.info(JSON.stringify(response.data))

    const imageStream = await axios.get(screenshotURL, {
      responseType: "stream",
    })
    return imageStream
  } catch (err) {
    console.error("\nError while taking the screenshot", err)
    throw err
  }
}

const sendEmail = async (receiversEmail) => {
  try {
    let mailerConfig = {
      host: "smtp.gmail.com",
      port: 587,
      secure: false, // true for 465, false for other ports
      auth: {
        user: "<GMAIL_ID>", // user
        pass: "<APP_PASSWORD>", // password
      },
    }

    let transporter = nodemailer.createTransport(mailerConfig)

    const imageStream = await takeScreenshot()

    const imagePath = path.join(__dirname, "..", "output", "screenshot.png")
    imageStream.data
      .pipe(fs.createWriteStream(imagePath))
      .on("finish", () => {
        // send mail with defined transport object
        let info = await transporter.sendMail({
          from: "<SENDER'S EMAIL ADDRESS>", // sender address
          to: `${receiversEmail}`, // list of receivers
          subject: "Screenshot requested", // Subject line,
          attachment: [
            {
              filename: imagePath,
              content: imageBuffer,
              encoding: "base64",
            },
          ],
          text: "Hello! find the screenshot that you requested attached", // plain text body
          html: "<b>Hello! find the screenshot that you requested attached</b>", // html body
        })
      })
      .on("error", (err) => {
        console.error("Stream closed with following error: ", err)
      })
    return true
  } catch (err) {
    console.error("\nError in sending the email", err)
    throw err
  }
}

module.exports = {
  sendEmail,
}

PS. If you want to use your Gmail account then you need to generate an app password. More details can be found here

❤ Like this blog post? Do follow me on Twitter

Do checkout my other blog posts:

Opinionated guide on tweaking VS Code for productivity

Neeraj Sewani — Wed, 06 Oct 2021 04:03:32 +0000

1). Be afraid of Windows: If you are on Windows then on one beautiful day you would be presented with a blue screen and you may not see your good old Windows again and the data. So, for that sunny day sync up all the setting, extensions and themes with your account.

2). Bid goodbye to the default theme: The default theme is okay but the theme can be better. Most common ones are Night Owl but I personally use SynthWave '84. To get a theme you need to download the individual themes from the extensions and then open command palette using ctrl + shift + p and then select the downloaded theme from there.

3). Knowing VS Code: Many times it has happened with me that I needed to search for "VS Code shortcut to show/hide the pane in the left". The point it that let's just know what different panes are called.

4). Get focused: Changing the position of the activity and the sidebar to the right side form the left. By doing this the editor doesn't shift on opening/closing(Ctrl + B) the sidebar as it does when they are in the left hence less annoyance.

5). Cut the clutter: If you see your Task bar, you would see a lot of junk there. Let's clear the clutter out by right clicking on the task bar and keeping the thing there that you really need there.

6). The minimap controversy: Most people out there would tell you to turn it off and give the most obvious reason that you don't need it but let me sell you an argument here. You can travel through a large file a lot faster in fewer clicks without scrolling a million times.

7). Clone a repo: Old days of cloning a repo and then opening it in the editor are gone. Clone a repo directly form command palette (ctrl + shift + p).

8). Good fonts, bad ligatures: Getting fancy with a better font is nice but enabling ligature(It basically means to combine different symbols into one eg. === turns into ≡) to make code resembles with Math or English which I believe is absurd and takes away the uniqueness of programming. It is as absurd as writing '8' in place of 'B' or 'o' as '0' in English.

If you want to enable it(why??) then download a font that support it, one of them it Fira code and watch the video below to enable it. May require a reload.

9). Extensions- VS code Samaritans: Have all of 'em, miss none

Auto Close Tag
Auto Rename Tag
Better Comments
Code Spell Checker
Color Highlight
Git Lens
Image Preview
Live Server
Import Cost
Markdown Preview Enhanced
Prettier
Thunder Client
Git Graph

10). 2 cents on Git lens: Get this extension and blame your colleagues with conviction as it shows the author's name on every line of the code. Putting it aside, we all once have felt the need to compare a file with a different version in a different commit or branch, here git lens really comes handy.
Press the previous revision button once to compare with the last commit, press it with "alt" to see all the commit of same branch or even compare with commits of different branch(first option).

11). 2 cents on Thunder Client: While switching between Postman and the editor zillion times is great muscle exercise but having an extension that does the same thing in the editor can be relaxing.

12). Mighty Tabs: Do you scroll to different files in the editor section and are frustrated? Not anymore, you can wrap all the tabs so that they all are visible altogether. I prefer to keep the tabs to 2 lines to avoid the clutter.

Also, you can close a tab using ctrl + w and reopen it using ctrl + shift + t same shortcuts as chromium based browsers which comes very handy.

13). AI powered settings search bar: How many times have you given up changing a setting just because you don't know what it is called? Many times maybe the answer but maybe not anymore. VS Code setting's search bar is AI powered and can process natural language. For eg. searching for "setting to zoom into the editor" pops up the option of "Mouse Wheel Zoom", which is also the Segway to the next point.

14). Zoom deep into the editor: It is one of the most common setting yet very useful. It is very handy if you are moving the same VS Code window around among different sizes of monitors.

15). Snippets: Now most of you have used snippets provided by the extensions that you have but we can have custom snippets that can be customizable for different languages. In JS you might have dealt with writing console.log/info/error entirely without snippets but this can be put into a snippet and called by a prefix.

16). Pain in the a** Preview: There is a setting enabled by default called "Preview Editor" which makes you click twice to open a file. Turn it off!!.

17). Useless Open Editors: Another setting that comes by default enabled is "Open Editors", can you see a useless space taking pane in the top of your "Sidebar"? yes, that's it. Turn it off and get yourself some space.

Deploy Twitter bot on Heroku

Neeraj Sewani — Mon, 31 May 2021 13:08:15 +0000

In part 1 of this post, we have made a Twitter bot that greets followers and people who like our tweets but it was dependent on our local machine. So, in this post let's get rid of it and push it on Heroku.

Heroku bestow the functionality to make a Node app a Worker which would run continuously as opposed to a Web app that would be put to sleep if not visited for a brief period of time.

Step-0: Make a Heroku app, then we can deploy the app using either Heroku CLI or attaching Github to it. Here, I am gonna use Heroku CLI.

Step-1: Add a Procfile to make our app a Worker. Simply make a file with the name Procfile add it to the root and add the following code in it:

worker: npm start

And modify the start script to "start": "node bot.js"

Step-2: Push the app to heroku by following the steps:

Install Git form here
Install Heroku CLI from here

Commit your changes using Git


  cd myapp
  git init
  Initialized empty Git repository in .git/
  git add .
  git commit -m "My first commit"

Then create a Heroku app using heroku create twitter-bot-2021, this would also add a remote to you repository. Check the remote using git remote -v. This should show a Herko remote.
Now open the app by login in to the Heroku in order to put environment variables there as we cannot push .env file for security reasons.
Get the app URL by opening up the app using open app option from inside the app dashboard.
Then go to settings and click on Reveal Config Vars and insert the environment variables present in your .env file.
Now, finally deploy the code to Heroku using git push heroku master:main, if you are on the main branch in your local git repo then simply do git push heroku main.
Checkout these docs related to deployment in case you are stuck somewhere.
Check the dashboard of the app, it should show a successful build in green.
Now, finally check the bot by following it form some other account, you should get the DM from the bot.

And you have done it.

Twitter bot in Node.Js

Neeraj Sewani — Mon, 31 May 2021 12:06:03 +0000

I bet you have wondered to automate some stuff on your social media, let's do some then.

With this bot we are going to DM anyone who follows us or likes our tweet. Here we are going to make use of the Account Activity API in order to consume follow and like events from our account.

Section-0: Environment Setup

STEP-0: Make a Twitter account( or use an existing one) and add an app to that by going to the following URL
P.S. I would recommend to make a separate account to deploy the bot on.

STEP-1: Go to app and get following from keys and tokens section.

API key, secret, Bearer Token, Access token and secret.
Enable 3 legged authentications to add callback URL for webhook. (A callback URL is required to register a webhook in order to consume events that would happen on our app, read more about it here)
Set up the dev environment by going to Products -> Dev Environments form the sidebar and name all of them dev.
Give Read, Write, and Direct Messages permissions form the App permissions section.

STEP-2: Install the following dependencies to use the Twitter APIs in Node.js

twitter-lite
dot-env
twitter-webhook
express
body-parser

STEP-3: Download and install ngrok to get a HTTPS URL for our local ports and to check the bot working before deploying it to Heroku.

Section-1: Securing API keys

STEP-0: First, make bot.js file as entry point to our app. Then, by making use of twitter-lite npm package we would interact with Twitter endpoints.

STEP-1: Now, open ngrok and get a HTTPS URL for the local port 3004 and paste the URL in the .env file along with the tokens obtained in Section 1.

STEP-2: Make a .env file to put all the sensitive info form Section-0

API_KEY=xIrIqvysinxgrSPm0Ir1D0cIF
API_SECRET_KEY=DjIMhoxJlRCxbgtw1zXdiGuKE4IjmjlAmnvB6orus24jbzmNof
ACCESS_TOKEN=1017764619337646080-3FrdsqhBLgVGDRAPEiy2a3fI7bY8Tv
ACCESS_TOKEN_SECRET=sRRfI6osPcvnwoHTY8UIA8y2hsI40kMltauL4xspAuMfr
ENVIRONMENT=dev
ROUTE=/callback-url
BEARER_TOKEN=AAAAAAAAAAAAAAAAAAAAACXW7gAAAAAA%2BbYioHfLTHR7Mf%2FnkpApHx1%2B%2FH0%3D5I7kLqCm5ejYNp5XoG8SbR96YoWxP3Po1J1RhyHwgPwj8E4rr8
SERVER_URL=https://fbc5f7e2c77f.ngrok.io

P.S. This makes the heading of the section ironic but anyway move ahead, there is lots to cover. Have already changed the tokens BTW.

STEP-3: Make a config.js for reading environment variables.

module.exports.twitter = {
  consumer_key: `${process.env.API_KEY}`,
  consumer_secret: `${process.env.API_SECRET_KEY}`,
  access_token_key: `${process.env.ACCESS_TOKEN}`,
  access_token_secret: `${process.env.ACCESS_TOKEN_SECRET}`,
}

module.exports.webhooks = {
  serverUrl: `${process.env.SERVER_URL}`,
  route: `${process.env.ROUTE}`,
  consumerKey: `${process.env.API_KEY}`,
  consumerSecret: `${process.env.API_SECRET_KEY}`,
  accessToken: `${process.env.ACCESS_TOKEN}`,
  accessTokenSecret: `${process.env.ACCESS_TOKEN_SECRET}`,
  environment: `${process.env.ENVIRONMENT}`,
}
module.exports.webhooksUserActivity = {
  accessToken: `${process.env.ACCESS_TOKEN}`,
  accessTokenSecret: `${process.env.ACCESS_TOKEN_SECRET}`,
}

Section-2: Code

STEP-0: In the following code we get the user id of our account and then register a webhook using twitter-webhooks package to receive follow and _ favorite tweet_ events on our account. Paste the following code in bot.js.

require('dotenv').config()

const express = require('express')
const Twitter = require('twitter-lite')
const twitterWebhooks = require('twitter-webhooks')

const { twitter, webhooks, webhooksUserActivity } = require('./config')

const twitterLiteClient = new Twitter({ ...twitter })

const app = express()
app.use(express.json())
app.use(express.urlencoded({ extended: true }))


const userActivityWebhook = twitterWebhooks.userActivity({
  ...webhooks,
  app,
})

;(async function () {
  //  get user id
  const result = await twitterLiteClient.get('users/lookup', {
    screen_name: '<twitter id of your app>',
  })
  const userId = result[0].id_str

  //     get webhooks
  const webhooksResult = await userActivityWebhook.getWebhooks()

  if (webhooksResult.environments[0].webhooks.length !== 0) {
    //  unregister earlier webhook
    const webhookId = webhooksResult.environments[0].webhooks[0].id
    await userActivityWebhook.unregister({
      webhookId,
    })
  }

  //Register your webhook url - just needed once per URL
  await userActivityWebhook.register()

  //Subscribe for a particular user activity
  userActivityWebhook
    .subscribe({
      ...webhooksUserActivity,
      userId,
    })
    .then(function (userActivity) {
      userActivity
        .on('follow', data => console.log(userActivity.id + ' - follow'))
        .on('favourite', data =>
          console.log(userActivity.id + 'favourite')
        )
    })
    .catch(console.error)
})()
app.listen(process.env.PORT || 3004)

STEP-1: Now, once we have attached the webhooks and started listening for the events, let's do some activity on our account using the twitter-lite package. Here I am greeting the new follower or the user who like a tweet by sending them a DM.

userActivityWebhook
    .subscribe({
      ...webhooksUserActivity,
      userId,
    })
    .then(function (userActivity) {
      userActivity
        .on('follow', async data => {
          const followerName = data.source.screen_name
          const followerId = data.source.id

          console.log(`\n ${followerName} followed you \n`)
          try {
            await twitterLiteClient.post('direct_messages/events/new', {
              event: {
                type: 'message_create',
                message_create: {
                  target: {
                    recipient_id: followerId,
                  },
                  message_data: {
                    text: `Hey ${followerName}! Thanks for following. You are awesome`,
                  },
                },
              },
            })
          } catch (err) {
            console.error(err)
          }
        })
        .on('favorite', async data => {
          console.log(JSON.stringify(data))
          const followerName = data.user.screen_name
          const followerId = data.user.id_str

          console.log(`\n ${followerName} liked a tweet\n`)

          try {
            await twitterLiteClient.post('direct_messages/events/new', {
              event: {
                type: 'message_create',
                message_create: {
                  target: {
                    recipient_id: followerId,
                  },
                  message_data: {
                    text: `Hey ${followerName}! Thanks for liking the tweet`,
                  },
                },
              },
            })
          } catch (err) {
            console.error(err)
          }
        })
    })
    .catch(console.error)

STEP-3: Run node bot.js to run it locally and do follow the bot from your personal account and you would get a DM form the bot.

P.S.: Sad part is that with a free account only 5 DMs can be sent in a 24 hour window. But a plethora of other things can be done with a much larger cap, here is the API reference

High five on getting it this far.

Follow part 2 of the tutorial to deploy it on Heroku.