DEV Community: Andreas

Why and how I built an Email Permutator

Andreas — Tue, 25 Jul 2023 20:42:46 +0000

Sometimes it takes a while to automate the trivial tasks … 10 years back, I started working as a marketer in the B2B software niche. I was confronted with email marketing, contact & and newsletter messaging from day one. The fact that I have an academic background in mechanical engineering and hence automation and programmatic approaches have always been a strong part of the way I work, make me question even more why it has taken me 10 years to automate something that I have solved manually over and over again: email permutation.

What is email permutation?

Email permutation is the variation of a person’s first- and lastname, combined with a TLD in order to find their email address. The basic assumption is that business email addresses follow commonly used syntax patterns, like the following:

johnsmith@example.com
john.smith@example.com
jsmith@example.com
j.smith@example.com
johns@example.com
john.s@example.com
js@example.com
j.s@example.com
smithjohn@example.com
smith.john@example.com
smithj@example.com
smith.j@example.com
sjohn@example.com
s.john@example.com
sj@example.com
s.j@example.com
john-smith@example.com
j-smith@example.com
john-s@example.com
j-s@example.com
smith-john@example.com
smith-j@example.com
s-john@example.com
s-j@example.com
john_smith@example.com
j_smith@example.com
john_s@example.com
j_s@example.com
smith_john@example.com
smith_j@example.com
s_john@example.com
s_j@example.com

converting john smith and example.com into variables makes the list look like this:

${firstname}@${domain}
${firstname}${lastname}@${domain}
${firstname}.${lastname}@${domain}
${firstLetterFirstname}${lastname}@${domain}
${firstLetterFirstname}.${lastname}@${domain}
${firstname}${firstLetterLastname}@${domain}
${firstname}.${firstLetterLastname}@${domain}
${firstLetterFirstname}${firstLetterLastname}@${domain}
${firstLetterFirstname}.${firstLetterLastname}@${domain}
${lastname}${firstname}@${domain}
${lastname}.${firstname}@${domain}
${lastname}${firstLetterFirstname}@${domain}
${lastname}.${firstLetterFirstname}@${domain}
${firstLetterLastname}${firstname}@${domain}
${firstLetterLastname}.${firstname}@${domain}
${firstLetterLastname}${firstLetterFirstname}@${domain}
${firstLetterLastname}.${firstLetterFirstname}@${domain}
${firstname}-${lastname}@${domain}
${firstLetterFirstname}-${lastname}@${domain}
${firstname}-${firstLetterLastname}@${domain}
${firstLetterFirstname}-${firstLetterLastname}@${domain}
${lastname}-${firstname}@${domain}
${lastname}-${firstLetterFirstname}@${domain}
${firstLetterLastname}-${firstname}@${domain}
${firstLetterLastname}-${firstLetterFirstname}@${domain}
${firstname}_${lastname}@${domain}
${firstLetterFirstname}_${lastname}@${domain}
${firstname}_${firstLetterLastname}@${domain}
${firstLetterFirstname}_${firstLetterLastname}@${domain}
${lastname}_${firstname}@${domain}
${lastname}_${firstLetterFirstname}@${domain}
${firstLetterLastname}_${firstname}@${domain}
${firstLetterLastname}_${firstLetterFirstname}@${domain}

Since my business also provides an email validation service (emailvalidation.io), I have published the email permutator on the website of our service.

What can be done with the retrieved list

The nature of the list above implies that max one email address of the list is actually valid. Sending emails to all addresses from the list will drastically destroy your reputation and will drive future email campaigns to the spam folder of your recipients.

Hence, the real magic starts you put the retrieved list of email addresses into an email validation service as a second step. This may provide you with a list of valid email addresses of contacts that is valid and can be targeted with your next email campaign. Of course you should always keep in mind to get people’s consent before adding them to your email campaign.

The most common email validation problems

Andreas — Thu, 01 Dec 2022 15:26:57 +0000

Introduction

Email validation can be a little confusing, but it's an important part of your email marketing strategy. Why? Email validation helps ensure that your emails reach the right person and avoid being marked as spam by their inboxes. It also allows you to track how many people are opening and clicking on your messages so that you can make changes to improve them over time.

Not validating your email.

Email validation is the process of confirming that an email address is valid. Email validation helps you determine if an email address is valid, and whether it will be delivered to the intended recipient.

A common misconception is that a “no-reply” or invalid email address means that no one can reply to your message. However, this isn't always true—it may sometimes mean that mail sent to this address will bounce back with an error message because it's unknown or not working properly (and you might still get some kind of automated response).
If you're sending out emails to a list of contacts and want to ensure that each person receives your communication, make sure all addresses are validated before sending out any messages!

Not tracking and analyzing your results

Tracking and analyzing the results of your emails is critical to growing your business.
This can be done manually or using a tool like Mailchimp. It’s important to keep track of open rates, click rates and unsubscribes so you can identify trends in your audience and adjust your strategy accordingly.
For example, if you notice that people are opening most of their emails in the morning but not clicking on them at all, then maybe it’s time to try sending an email at another time of day. If your email validation has shown that people aren’t clicking on certain subject lines or content types, maybe it's time for some adjustments there as well!

Using the wrong email validation tools

Email validation is a complex process. It's not simply a matter of checking whether an email address exists or not—it's also important to check the syntax and make sure that domains are valid, among other things.
You should use the right tool for the job. A general web service like Google Apps will not be able to handle all of these tasks, so don't try it! You need an email validation tool specifically built for this purpose.
If you do decide to use a web service that isn't designed specifically for email validation (such as Google Apps or emailvalidation.io then make sure it does at least two out of three: checks if an address exists; checks if domains are valid; checks whether domains are syntactically correct (and only do this if there's no way around it).

Not correcting typos in emails

The most common email validation problems are simple typos. Sometimes you hit the wrong button, or maybe you're just so excited about something that your fingers fly over the keyboard without checking to see if everything is spelled correctly.
Don't worry—all hope isn't lost! There are ways to avoid typos in emails and even fix them after they've been sent out.
To prevent these mishaps from happening, try taking a few deep breaths before sending an email out to your coworkers or clients. This will give you time to proofread it and catch any errors before they reach their intended destination. If it's more of an emergency situation (e.g., someone else needs access urgently), try asking someone else for help! A fresh pair of eyes can often spot mistakes that we're too close to see ourselves.
If you do end up sending out an email with spelling or grammar errors, don't panic; there's still hope for redemption! Just send another message apologizing for any mistakes made and stating how much work went into getting the message corrected as soon as possible (eagerly awaiting your response).

Adding yourself to spam lists

If you use email validation tools that are not compatible with your email provider, it could result in your emails being filtered as spam.
Adding yourself to spam lists is also a common mistake. Even if you’re sending emails through a reputable service, there’s no guarantee that everyone on your mailing list wants to get them.
Finally, never send emails to people who have not consented to receive them.

Email validation ensures that your emails reach the right person and can help you improve your open rates and click-through rates

Email validation is important for improving your email marketing performance. Because it's so easy to get an incorrect or outdated email address, the benefits of email validation go beyond just ensuring that messages reach the right person. It can also help you improve your open rates and click-through rates, as well as deliverability and bounce rate.

Conclusion

Don’t let email validation slip away from you. Email validation is an important part of any marketing strategy, and it’s easy to do. You can use APIs or create your own validation rules using regular expressions. If you keep these seven pitfalls in mind when validating emails, you’ll be well on your way to better open and click-through rates!

Web Scraping with PHP: Building a Competitor Price Monitoring Tool

Andreas — Mon, 18 Jan 2021 13:42:32 +0000

Depending on your business's niche and market, adjusting your services and prices has to come along with taking your competitors into account.
In a lot of companies that I have seen, this is a manual task, that is completed once every quarter or at least every year.
In this PHP web scraping tutorial, we are going to build a tiny tool, that automizes this process. Of course, the tool will need further advancements, but is always about understanding the concepts, right? :)

Let's get started!

Prerequisites

We will need the following set of tools:

Web server with PHP
Composer
Guzzle - scraping client
PHP HTML Parser - as HTML Parser
A currency parser

Download composer here to download the composer and follow the install instructions.

After composer has successfully been installed, install guzzle via composer:

composer require guzzlehttp/guzzle

Next, let's install our HTML parser:

composer require paquettg/php-html-parser

Finally, we add the currency parser to our project:

composer require mcuadros/currency-detector dev-master

Building the scraper

As we want to build a competitor price monitoring tool, let's say that this product URL is our own:

https://www.allendalewine.com/products/11262719/diplomatico-reserva-exclusiva

As a competitor page, we select the following:

https://www.winetoship.com/diplomatico-rum-reserva-exclusiva.html

Next, we have to define the CSS-Selectors that contain the price information.

For our "own" website, the selector is .sale-price.currency. Going through the same process for the competitor, the selector is .less-price .o_price span.

Putting the pieces together, we end up with the following script:

<?php
require 'vendor/autoload.php';

use \GuzzleHttp\Client;
use \PHPHtmlParser\Dom;
use \CurrencyDetector\Detector;


$productPairs = [
    'rum' => [
        'own' => [
            'url' => 'https://www.allendalewine.com/products/11262719/diplomatico-reserva-exclusiva',
            'selectorPath' => '.sale-price.currency'
        ],
        'competitor1' => [
            'url' => 'https://www.winetoship.com/diplomatico-rum-reserva-exclusiva.html',
            'selectorPath' => '.less-price .o_price span'
        ]
    ]
    # you can add as many product pairs as you wish
];

$detector = new Detector();

$comparison = [];

foreach ($productPairs as $productName => $pair) {


    foreach($pair as $provider => $product) {

        $client = new Client();
        $parser = new Dom;

        $request = $client->request('GET', $product['url']);
        $response = (string) $request->getBody();
        $parser->loadStr($response);
        $price = $parser->find($product['selectorPath'])[0];
        $priceString = $price->text;

        $fmt = new NumberFormatter( 'en_US', NumberFormatter::CURRENCY );

        $comparison[$productName][$provider] = [
            'currency' => $detector->getCurrency($priceString),
            'amount' => $detector->getAmount($priceString),
        ];     

    }
}

echo json_encode($comparison);

You can add as many product and competitor entities as you like. The scraper then loops through all products and competitors and fetches the HTML-Markup. Our DOM-Parser then extracts the related elements from the HTML. Finally, the currency detector parses the price string into a comparable and normalized format.

I used the following PHP web scraping tutorial to create this scraper.

My attempt to build a worldwide zip code data set

Andreas — Fri, 02 Oct 2020 15:40:35 +0000

I always thought getting worldwide postal codes by myself was an easy task because postal codes seem to be nothing more than a simple shortcode that is publicly available. I quickly realized this was not the case, because:

There is no single source of truth
Most sources were incomplete
Data was very often presented in a very unstructured way

After doing some general research, I soon understood, that the reason for the problems above had their origin in the history of postal codes. Each country has a different format, area granularity, and way of structuring postal codes as a whole.

I first tried to scrape Wikipedia with the following code. For this post, I will use the example of Austria.

For this, I a small python script.
Before running it make sure to install all dependencies:

pip3 install lxml
pip3 install requests,
pip3 install bs4


import requests
from bs4 import BeautifulSoup

url = 'https://en.wikipedia.org/wiki/List_of_postal_codes_in_Austria'

# fire GET request
response = requests.get(url)

# parse content
content = BeautifulSoup(response.text, 'lxml')

# get postal codes

postcodes = [
    postcode.text for postcode in content.find_all('li')
    if ' - ' in postcode.text
]

# filter edge cases

postcodes = [
    postcode.split()[0] for postcode in postcodes 
    if len(postcode.split()) == 3 or
    len(postcode.split()) == 4
]

# write output to file
with open('at_postcodes.txt', 'a') as f:
    for postcode in postcodes:
        f.write(postcode + '\n')

The obtained data set and the related approach might be enough for some use cases, but since I wanted to get global postal code data, I was not satisfied.

I live in Austria and realized very quickly that the data that I have just scraped is not complete (some postal codes are missing). Considering the time it took my to build the parser and the fact that I would have to adapt it for every single data source (adaptions are even needed across Wikipedia since every article is written differently), I decided to give up.

This was the moment I gave up and started to look for ready-to-use solutions:

I hope this article will save you some time, in case you are trying to achieve the same.

Scraping Soccer Data with Nodejs and Puppeteer

Andreas — Fri, 11 Sep 2020 13:27:57 +0000

I was recently in a situation of needing sports data - soccer data in particular. I am writing this post, as I had to overcome some initial problems. You should be able to follow my thoughts and my path that lead me to a solution.

For this tutorial, I came across fleshscore.com, a website that provides plenty of leagues and covers fixtures and live matches.

I started with the following basic script:

const axios = require('axios');
    //performing a GET request
axios.get('https://www.flashscore.com/')
    .then(response => {

    //handling the success
    const html = response.data;
    console.log(response.data);
})
//handling error
.catch( error => {
  console.log(error);
});

To investigate what is being returned by the script, I forwarded all returned content into a test.html file.

node scraper.js > test.html

After I had opened the HTML-file inside my browser, I quickly realized that all match information, that was shown on the original website, was missing. This was not a big surprise, as I expected the content to be rendered by javascript.

As the script above is written in nodejs, I started to play around with puppeteer, which is a node library that provides a high-level API to control headless Chrome or Chromium.

After some time, I ended up with the following piece of code:

const puppeteer = require ('puppeteer');

//initiating Puppeteer
puppeteer
    .launch ()
    .then (async browser => {

        //opening a new page and navigating to Fleshscore
        const page = await browser.newPage ();
        await page.goto ('https://www.flashscore.com/');
        await page.waitForSelector ('body');

        //manipulating the page's content
        let grabMatches = await page.evaluate (() => {
        let allLiveMatches = document.body.querySelectorAll ('.event__match--oneLine');

        //storing the post items in an array then selecting for retrieving content
        scrapeItems = [];
        allLiveMatches.forEach (item => {

            let postDescription = '';
                try {
                    let homeTeam = item.querySelector ('.event__participant--home').innerText;
                    let awayTeam = item.querySelector ('.event__participant--away').innerText;
                    let currentHomeScore = item.querySelector('.event__scores.fontBold span:nth-of-type(1)').innerText;
                    let currentAwayScore = item.querySelector('.event__scores.fontBold span:nth-of-type(2)').innerText;
                    scrapeItems.push ({
                        homeTeam: homeTeam,
                        awayTeam: awayTeam,
                        currentHomeScore: currentHomeScore,
                        currentAwayScore: currentAwayScore,
                    });
                } catch (err) {}

            });
            let items = {
                "liveMatches": scrapeItems,
            };
            return items;
        });
        //outputting the scraped data
        console.log (grabMatches);
        //closing the browser
        await browser.close ();
    })
    //handling any errors
    .catch (function (err) {
        console.error (err);
    });

Now I ran the script again with the following command:

node scraper.js

As you can see I retrieved a beautiful list of JSON data.
Now, of course, there is plenty of work that could be spent to sort the data by the league, country, etc. etc.

For my use case, this snippet was enough. If you aim for more serious scraping, you may as well pick a general sports- or soccer API (I.e. sportdataapi.com, xmlsoccer.com.

Happy Scraping :-)

Introduction to Web Scraping With JavaScript and Node.js

Andreas — Wed, 09 Sep 2020 13:42:04 +0000

With the massive increase in the volume of data on the Internet, this technique is becoming increasingly beneficial in retrieving information from websites and applying them for various use cases. Typically, web data extraction involves making a request to the given web page, accessing its HTML code, and parsing that code to harvest some information. Since JavaScript is excellent at manipulating the DOM (Document Object Model) inside a web browser, creating data extraction scripts in Node.js can be extremely versatile. Hence, this tutorial focuses on javascript web scraping.

In this article, we’re going to illustrate how to perform web scraping with JavaScript and Node.js.

We’ll start by demonstrating how to use the Axios and Cheerio packages to extract data from a simple website.

Then, we’ll show how to use a headless browser, Puppeteer, to retrieve data from a dynamic website that loads content via JavaScript.

What you’ll need

Web browser
A web page to extract data from
Code editor such as Visual Studio Code
Node.js
Axios
Cheerio
Puppeteer

Ready?

Let’s begin getting our hands dirty…

Getting Started

Installing Node.js

Node.js is a popular JavaScript runtime environment that comes with lots of features for automating the laborious task of gathering data from websites.

To install it on your system, follow the download instructions available on its website here. npm (the Node Package Manager) will also be installed automatically alongside Node.js.

npm is the default package management tool for Node.js. Since we’ll be using packages to simplify web scraping, npm will make the process of consuming them fast and painless.

After installing Node.js, go to your project’s root directory and run the following command to create a package.json file, which will contain all the details relevant to the project:

npm init

Installing Axios

Axios is a robust promise-based HTTP client that can be deployed both in Node.js and the web browser. With this npm package, you can make HTTP requests from Node.js using promises, and download data from the Internet easily and fast.

Furthermore, Axios automatically transforms data into JSON format, intercepts requests and responses, and can handle multiple concurrent requests.

To install it, navigate to your project’s directory folder in the terminal, and run the following command:

npm install axios

By default, NPM will install Axios in a folder named node_modules, which will be automatically created in your project’s directory.

Installing Cheerio

Cheerio is an efficient and lean module that provides jQuery-like syntax for manipulating the content of web pages. It greatly simplifies the process of selecting, editing, and viewing DOM elements on a web page.

While Cheerio allows you to parse and manipulate the DOM easily, it does not work the same way as a web browser. This implies that it doesn’t take requests, execute JavaScript, load external resources, or apply CSS styling.

To install it, navigate to your project’s directory folder in the terminal, and run the following command:

npm install cheerio

By default, just like Axios, npm will install Cheerio in a folder named node_modules, which will be automatically created in your project’s directory.

Installing Puppeteer

Puppeteer is a Node library that allows you to control a headless Chrome browser programmatically and extract data smoothly and fast.

Since some websites rely on JavaScript to load their content, using an HTTP-based tool like Axios may not yield the intended results. With Puppeteer, you can simulate the browser environment, execute JavaScript just like a browser does, and scrape dynamic content from websites.

To install it, just like the other packages, navigate to your project’s directory folder in the terminal, and run the following command:

npm install puppeteer

Scraping a simple website

Now let’s see how we can use Axios and Cheerio to extract data from a simple website.

For this tutorial, our target will be this web page. We’ll be seeking to extract the number of comments listed on the top section of the page.

To find the specific HTML elements that hold the data we are looking for, let’s use the inspector tool on our web browser:

As you can see on the image above, the number of comments data is enclosed in an <a> tag, which is a child of the <span> tag with a class of comment-bubble. We’ll use this information when using Cheerio to select these elements on the page.

Here are the steps for creating the scraping logic:

Let’s start by creating a file called index.js that will contain the programming logic for retrieving data from the web page.
Then, let’s use the require function, which is built-in within Node.js, to include the modules we’ll use in the project.
```
  const axios = require('axios');
  const cheerio = require('cheerio');
```
Let’s use Axios to make a GET HTTP request to the target web page.

Here is the code:
```
axios.get('https://www.forextradingbig.com/instaforex- 
broker-review/')
   .then(response => {
      const html = response.data;      
   })
```
Notice that when a request is sent to the web page, it returns a response. This Axios response object is made up of various components, including data that refers to the payload returned from the server.

So, when a GET request is made, we output the data from the response, which is in HTML format.
Next, let’s load the response data into a Cheerio instance. This way, we can create a Cheerio object to help us in parsing through the HTML from the target web page and finding the DOM elements for the data we want—just like when using jQuery.

To uphold the infamous jQuery convention, we’ll name the Cheerio object $.

Here is the code:
```
const $ = cheerio.load(html);
```
Let’s use the Cheerio’s selectors syntax to search the elements containing the data we want:
```
const scrapedata = $('a', '.comment-bubble').text()
console.log(scrapedata);
```
Notice that we also used the text() method to output the data in a text format.
Finally, let’s log any errors experienced during the scraping process.
```
.catch( error => {
console.log(error);
}); 
```
Here is the entire code for the scraping logic:

If we run the above code with the node index.js command, it returns the information we wanted to scrape from the target web page.

Here is a screenshot of the results:

It worked!

Scraping a dynamic website

Now let’s see how you can use Puppeteer to extract data from a dynamic website.

For this example, we’ll use the ES2017 asynchronous async/await syntax to work with promises comfortably.

The async expression implies that a promise will be returned. And the await expression makes JavaScript wait until that promise is resolved before executing the rest of the code. This syntax will ensure we extract the webpage’s content after it has been successfully loaded.

Our target will be this Reddit page, which uses JavaScript for rendering content. We’ll be seeking to extract the headlines and descriptions found on the page.

To find the specific HTML elements that hold the data we are looking for, let’s use the inspector tool on our web browser:

As you can see on the image above, each post is enclosed in a Post class, amongst other stuff. By examining it closely, we find that each post title has an h3 tag, and each description has a p tag. We’ll use this information when selecting these elements on the page.

Here are the steps for creating the scraping logic:

Let’s start by creating a file called index.js that will contain the programming logic for retrieving data from the webpage.
Then, let’s use the require function, which is built-in within Node.js, to import Puppeteer into our project.
```
const puppeteer = require ('puppeteer');
```
Let’s launch Puppeteer. We’re actually launching an instance of the Chrome browser to use for accessing the target webpage.
```
puppeteer.launch()
```
Let’s create a new page in the headless browser. Since we’ve used the await expression, we’ll wait for the new page to be opened before saving it to the page variable.

After creating the page, we’ll use it for navigating to the Reddit page. Again, since we’ve used await, our code execution will pause until the page is loaded or an error is thrown.

We’ll also wait for the page’s body tag to be loaded before proceeding with the rest of the execution.

Here is the code:
```
.then (async browser => { 
const page = await browser.newPage ();  
await page.goto ('https://www.reddit.com/r/scraping/');     
await page.waitForSelector ('body');
```
After pulling up the Reddit page in Puppeteer, we can use its evaluate() function to interact with the page.

With the function, we can execute arbitrary JavaScript in Chrome and use its built-in functions, such as querySelector(), to manipulate the page and retrieve its contents.

Here is the code:
Let’s close the browser.
```
await browser.close ();
```
Finally, let’s log any errors experienced during the scraping process.
```
.catch (function (err) {
console.error (err);
});
```
Here is the entire code for the scraping logic:

If we run the above code with the node index.js command, it returns the information we wanted to scrape from the target web page.

Here is a screenshot of the results (for brevity, the results have been truncated):

It worked!

If you intend to use the above in production and make thousands of requests to scrape data, you’ll definitely get banned. In this scenario, rotating your IP addresses after every few requests can help you to stay below their radar and extract content successfully.

Therefore, connecting to a proxy service can help you to make the most of your scraping efforts. Importantly, with residential proxies, you can get around the scraping bottlenecks and harvest online data easily and fast.

In Puppeteer, you can easily connect to a proxy by passing one extra line of arguments when launching it:

puppeteer.launch({
    args: [ '--proxy-server=145.0.10.11:7866' ]
});

Conclusion

That’s how you can perform web scraping with JavaScript and Node.js. With such skills, you can harvest useful information from web pages and integrate them into your use case.

Remember that if you want to build something advanced, you can always check Axios, Cheerio, and Puppeteer documentation to assist you in getting your feet off the ground quickly.

Happy scraping!

How to Scrape Tripadvisor Reviews with Nodejs and Puppeteer

Andreas — Mon, 04 Nov 2019 15:22:39 +0000

Tripadvisor contains tons of useful local business reviews. However, the site’s API does not provide an endpoint to access them. Also, Tripadvisor uses a lot of javascript in the frontend, which makes it a little bit harder to scrape than a lot of other websites.

In this quick tutorial, we are going to walk through all steps to scrape the customer reviews from a tripadvisor local business page.

Why Puppeteer?

Let me quickly say a few words about why I decided to use Puppeteer for this project. As I mentioned above, Tripadvisor requires a full browser, because a lot of content requires javascript, in order to be rendered. That basically left me with two options: Selenium and Puppeteer. Throughout the last months, Puppeteer has evolved to be the more prominent solution, as it is noticeably faster.

Information we are going to scrape

For this tutorial, I have selected a random pizzeria in New York City. It has the following profile url:

https://www.tripadvisor.com/Restaurant_Review-g60763-d15873406-Reviews-Ortomare_Ristorante_Pizzeria-New_York_City_New_York.html

As you can see in the screenshot above, we are going to scrape the following pieces of information of each review:

• The rating
• Date of the review
• Date of the visit
• The review title
• Review text (we will have to expand it)

Getting started

Before we start extracting the code, we will have to install puppeteer on our local environment:

npm install puppeteer --save

The full code


/* Part 1 */

const puppeteer = require('puppeteer');

puppeteer.launch({ headless: true, args: ['--no-sandbox', '--disable-setuid-sandbox', '--window-size=1920,1080'] }).then(async browser => {

    const page = await browser.newPage();
    await page.goto("https://www.tripadvisor.com/Restaurant_Review-g60763-d15873406-Reviews-Ortomare_Ristorante_Pizzeria-New_York_City_New_York.html");
    await page.waitForSelector('body');

/* Part 2 */

    await page.click('.taLnk.ulBlueLinks');
    await page.waitForFunction('document.querySelector("body").innerText.includes("Show less")');

/* Part 3 */

var reviews = await page.evaluate(() => {
       
        var results = [];

        var items = document.body.querySelectorAll('.review-container');
        items.forEach((item) => {
            
            /* Get and format Rating */
            let ratingElement = item.querySelector('.ui_bubble_rating').getAttribute('class');
            let integer = ratingElement.replace(/[^0-9]/g,'');
            let parsedRating = parseInt(integer) / 10;

            /* Get and format date of Visit */
            let dateOfVisitElement = item.querySelector('.prw_rup.prw_reviews_stay_date_hsx').innerText;
            let parsedDateOfVisit = dateOfVisitElement.replace('Date of visit:', '').trim();

/* Part 4 */

            results.push({
                    rating: parsedRating,
                    dateOfVisit: parsedDateOfVisit,
                    ratingDate: item.querySelector('.ratingDate').getAttribute('title'),
                    title:  item.querySelector('.noQuotes').innerText,
                    content: item.querySelector('.partial_entry').innerText,
                
            });
            
        });
        return results;
    });
    console.log(reviews);        
    await browser.close();

}).catch(function(error) {
    console.error(error);
});

Let me walk through the parts of the code:

Part 1:

With these first lines, we launch puppeteer in headless mode and navigate to the profile page of the pizzeria. All following actions need the document body to be fully loaded. This is ensured by the last line of part 1.

Part 2:

As you can see above, not the entire review text is shown by default. Hence, we have to click on “More”, before scraping the actual content. This is executed by line 8. Again, the following code is only to be executed, once the click action has successfully been completed. This is ensured by the last line of part 2.

Part 3:

This is where the magic happens. We access the page-DOM and extract all desired information from each review that is visible on the page.

Rating:

By taking a closer look at the element, we can see that the ratings are made up from pseudo elements. However, there is a class on the element from which we can conclude the rating:

This review shows a 5/5 rating. We can calculate the given rating by extracting the number “50” from the string “bubble_50”, convert it to an integer, and divide it by 10.

Date of visit: The date of visit can be obtained quite easily. We simply select the element that contains the date and remove the substring “Date of visit:”.

Review title and content:

These can be simply extracted by simply getting the text from the related elements. No manipulation needed.

We have now successfully gathered all information.

Part 4:

We append all gathered information in an array array, which is then returned by the function.

Running the script in your console should return all ten results from the first page.

Possible improvements

The script above only returns the reviews that are shown on the first page. In order to obtain all available reviews, you have to paginate through all review pages. Each page contains up to 10 reviews. This can be achieved by clicking on the page links on the bottom of the page, like we did with the “show more” links.
When scraping the reviews of a bigger list of restaurants, I recommend using a puppeteer cluster. Make sure to limit the concurrency, so you are not spamming too much.
Also, your scraper might get blocked at some point. This is one of the major issues my current startup, a web scraping API, is trying to solve: zenscrape.com
Zenscrape also offers a great tutorial on how to get started with nodejs getting started with nodejs and puppeteer