Day21 of 100DaysOfCode

I have been coding web scraping scripts since Day 20.
Today, I learned a few typical methods for web scraping.

What I learned

• Using waiting method to ensure the webpage fully loads

// if you used `.click()`
await Promise.all([
    pointOutSomeElements[0].click(),
    page.waitForNavigation({ waitUntil: ["load", 'networkidle0']}), // This line is used as it is.
]);

// if you used `.goto()` 
await page.goto(url, {
    timeout: 0,
    waitUntil: 'networkidle2'
});

This code block is useful when you encounter an Error like the following:
Error: Execution context was destroyed, most likely because of a navigation.

Executing actions such as .click() or .goto() before the navigation is fully completed can lead to this error.
Therefore, it's important to wait for the webpage to finish navigating.

For more details, you can refer to the links below:

• Execution context was destroyed, most likely because of a navigation #3323
• Puppeteer Execution context was destroyed, most likely because of a navigation