DEV Community

Cover image for Productivity Hacks Using Puppeteer
sudarshan
sudarshan

Posted on

Productivity Hacks Using Puppeteer

TL:DR :

Puppeteer is one of the greatest tools available on the web in order to perform the automation regarding website or In-Browser Activities. Today, Instead of single huge chunk of code (which does nothing helpful other than only page screenshots ) I will try to explain the several tricks which you can use in your next project or any testing purposes.

πŸ‘» What is Puppeteer :

Puppeteer is browser automation tool which is used to automated activities on the internet using Headless Chromium Browser.

Puppteer uses the high level API provided by the google in order to interact with headless chrome. It provides high level of flexibility and extensibility in order to modify the default behavior of Puppeteer.


πŸš€ Launch It :

For launching Puppeteer, people normally use basic configuration, but here I will show some advance modification in order to launch Puppeteer in various modes

image

Here we are using main three options

  • Lauching Puppteer using headless flag as false , because it will show the activity of browser on the screen
  • Second flag is linux specific. I used it here, because if i excluded it then code will break on ubuntu 20.04
  • This option is pretty useful because it removes the tag from the browser window which is "* Google chrome is controlled by the automated software*"

Then all we are doing is launching it using launch() and then redirecting the page to google


πŸ€– Launching the Puppeteer in the Incognito Mode :-

Whenever we are doing automation, there is a point when you will be forced to use the incognito mode of the chrome. It is pretty helpful, if a site is preventing you from accessing the useful stuff, because they stored the cookie in you localStorage marking you as a robot and when you are accessing the stuff it is blocking your access

Then solution is here

image

Add this two line snippet while launching and Puppteer will open automatically in the incognito mode. COOL 8)


Handle Navigation :

Handling navigation is pretty important while you are extending the testing / automation to the multiple pages. If the navigation isn't handled properly then we may get this

Failed to execute. Mostly because context destroyed due to navigation
Enter fullscreen mode Exit fullscreen mode

To avoid this stuff handle navigation as follows

image

This is the example of performing the search on the google. Type the search term in the search box (with provided delay in order to avoid the bot detection) and then press Enter.

After it, wait until navigation completed and when the DOM content is loaded completely then proceed further.

Further to these, we can use two constraint to check whether the page is loaded completely or not

  • 'domcontentloaded' :- as soon as DOM loaded proceed
  • 'load' :- wait for all things load using Javascript and CSS files and images

😎 Interact with elements :

We have to click on the web in order to get info regarding something. We all Know that but Puppteer doesn't. Hence, forcing it to click on element is one of crucial functionality we all must know.

image

πŸ˜‚ You will use this a lot, because it allows the smooth scrolling up to the element which is to be clicked. If it is present at the bottom then instead of clicking it directly, it will first scroll up to the point and the click on it.


Evaluate things using $$eval() :

It is pretty useful, if the scenario occurs where you don't want to interact directly with DOM then we can use the eval() in order to get things from element
For ex. clicking on it, getting text content etc

image

using $$eval() over the native dom API's like

javascript document.getElementById("blah")

is preferred by many individual. Because, if we used eval() then Puppteer will efficiently searches through the DOM and intern manages all things internally but by using native API's we are adding additional load on puppteer. Although it is preferential

🎯 Final Thought :

Puppeteer is pretty useful for automated testing and other stuff. But, you must have to take care of detection. In this article, I touched over the few things (* Less than 1% *) which are important in day to usage. but, if you want to explore more please let me know in comments. I will love to hear your thoughts πŸ’œ

πŸ™ Thanks For Reading....

Top comments (13)

Collapse
 
0neilwe profile image
0neilwe

very good article,
how did you display your code like that ?

Collapse
 
sudarshansb143 profile image
sudarshan

Thanks !

carbon.now.sh/?bg=rgba%28171%2C195...

Thanks me again for this later if you liked it :)

Collapse
 
tiim profile image
Tim Bachmann

You should probably add the code as text in a code block as well, for accessibility and copy pasting :)

Thread Thread
 
sudarshansb143 profile image
sudarshan

Point is noted @tiim . I will take care of it next time

Collapse
 
hartley94 profile image
Martin Thuo

Thank you.

Collapse
 
toastsgithub profile image
Duanzm

what's the font of your code btw?

Collapse
 
sudarshansb143 profile image
sudarshan

It is source code pro for this post

Collapse
 
toastsgithub profile image
Duanzm

okay thank you bro, but I mean the fancy font in your carbon code block, it looks cute

Thread Thread
 
sudarshansb143 profile image
sudarshan

I don't know exactly which one i used, but i got that on runtime.

Anyways, Thanks for feedback

Collapse
 
carl0smore1ra profile image
Carlos Moreira

Nice tip, I use it to register my monthly hours using a csv from toggle entries.
This task was so boring and time consumed.

Collapse
 
sudarshansb143 profile image
sudarshan

Yup ! That's cool

If you are interacting with CSV take a look at here and thanks me later

dev.to/sudarshansb143/top-utility-...

Collapse
 
mehuge profile image
Mehuge

1st rule of programming. Sort out your indentation.

Collapse
 
sudarshansb143 profile image
sudarshan

Ok !