DEV Community: KoichiArai

Day 23 of 100DaysOfCode

KoichiArai — Thu, 03 Oct 2024 16:47:16 +0000

Hey! It's been a while since last entry.
Recently, I've been tackling an issue related to losing element references within a loop, and I finally found a solution that I'd like to share with you all.

What I wanted to achieve

Collect button elements on a specific webpage.
Click each buttons to navigate to another page.
After performing certain actions on the new page, go back to the first page.
Repeat Steps 2 and 3 until all buttons have been clicked.

Here's a simplified version of the initial code:

const someElements = await page.$$("xpath/ .//foo[@name='hoge']");

for (let i=0; i < someElements.length; i++) {
    await Promise.all([
        someElements[i].click(),
        page.waitForNavigation({ waitUntil: ["load", 'networkidle0'] }),
    /* There are particular processes and going back process */
    ]);
};

What error I encountered

After the forst loop iteration, the script wasn't able to locate the button elements. I kept getting the following error:
ProtocolError: Protocol error (DOM.describeNode): Cannot find context with specified id at <instance_members_initializer>

How I solved it

I resolved this issue by re-declaring the button elements list within the loop.
Here's the modified code:

const someElements1 = await page.$$("xpath/ .//foo[@name='hoge']");

for (let i=0; i < someElements.length; i++) {
    const someElements2 = await page.$$("xpath/ .//foo[@name='hoge']"); // added row
    await Promise.all([
        someElements2[i].click(),
        page.waitForNavigation({ waitUntil: ["load", 'networkidle0'] }),
    /* There are particular processes and going back process */
    ]);
};

I added re-declaration line because I assumed the original someElements reference was getting lost when navigating away from the initial page. After this change, the error was resolved. However, I'm not sure if this approach strictly adheres to best coding practices.

If you encounter an issue with .click() in a loop when navigating between pages, consider re-declaring the element within the loop itself. This simple change might save you a lot of troubleshooting time!

Day22 of 100DaysOfCode

KoichiArai — Sat, 07 Sep 2024 11:44:09 +0000

I'm still working on scraping tasks.
During these tasks, I've picked up a few techniques for specific situations, and I'm nothing them down for future reference.

What I learned

It's better to use the method .click() when the targeted element has an attribute like href="Javascript:~". This ensures the click action is properly triggered.
When you want to input text into a pre-filled textbox, clear it first by using:

await page.$eval("selector", element => element.value = "")

This will help avoid appending text to the existing value.

The method page.waitForTimeout doesn't exist. In fact, .waitForTimeout is deprecated and shouldn't be used.

Day21 of 100DaysOfCode

KoichiArai — Thu, 22 Aug 2024 16:33:56 +0000

I have been coding web scraping scripts since Day 20.
Today, I learned a few typical methods for web scraping.

What I learned

Using waiting method to ensure the webpage fully loads

// if you used `.click()`
await Promise.all([
    pointOutSomeElements[0].click(),
    page.waitForNavigation({ waitUntil: ["load", 'networkidle0']}), // This line is used as it is.
]);

// if you used `.goto()` 
await page.goto(url, {
    timeout: 0,
    waitUntil: 'networkidle2'
});

This code block is useful when you encounter an Error like the following:
Error: Execution context was destroyed, most likely because of a navigation.

Executing actions such as .click() or .goto() before the navigation is fully completed can lead to this error.
Therefore, it's important to wait for the webpage to finish navigating.

For more details, you can refer to the links below:

Day20 of 100DaysOfCode

KoichiArai — Sun, 18 Aug 2024 10:02:16 +0000

I've been working on building another program since Day19. (I had to focus on other coding tasks.)
During this coding session, I encountered a few problems and was able to solve some of them.

Challenges I faced

How to use page.waitForXPath(xpath) The answer to this issue was found here:

"page.waitForXPath is not a function" in 22.10.0

The correct usage is to use waitForSelector(xpath/${xpathexpression}) instead.

SyntaxError: Unexpected token 'export'
I ran into this error when I used the export statement in my function code. After trying various solutions, I couldn't figure out how to resolve it. So I decided to switch to using import instead of require in my main code.
TypeError: page.$x is not a function
This was the toughest issue I faced today. Unfortunately, neither ChatGPT nor Google provided a clear solution. i tried various suggestions I found, such as removing and reinstalling Puppeteer, checking the Puppeteer version, and cleaning up dependencies in package.json, but none of these worked. I still haven't figured out the true cause of this problem.
The instant solution to this issue is here:

refactor!: remove $x and waitForXpath #11782

Day19 of 100DaysOfCode

KoichiArai — Fri, 09 Aug 2024 02:11:34 +0000

It's hard to keep coding consistently, especially since I've been working overtime until 10p.m. lately.
(I know it's not great to make excuses, but I'm determined to push through.)
By the way, I added a **contextBridge** (by including a **preload.js** file) to the previous program.
However, it doesn't work and I have been trying to solve this issue.

What I did

Added a secure process

// index.js
const button = document.getElementById("button");
const receive = document.getElementById("receive");
const send = document.getElementById("send");

button.addEventListener('click', async () => {
    receive.textContent = await window.myAPI.buttonEvent(send.value);// Changed
});

// preload.js
const {ipcRenderer, contextBridge} = require('electron');

contextBridge.exposeInMainWorld('myAPI',{
    buttonEvent: (text) => ipcRenderer.invoke('two-way-com', text),
});

I'll continue working on solving this problem.

Day18 of 100DaysOfCode

KoichiArai — Thu, 01 Aug 2024 15:19:03 +0000

Today, I made a small program using two-way Inter-Process Communication (IPC).
Then, a defect occurred, and I struggled with it for an hour.
After all, the cause was a mere mistake...(embarrassing!)

What I did

Making small program to learn how two-way IPC behaves

What I learned

Extracting text from a textbox on GUI

Today's deliverables

// main.js
"use strict"
const { app, BrowserWindow, ipcMain} = require('electron');

function createWindow() {
    const mainWindow = new BrowserWindow({
        width: 400,
        height: 300,
        title: 'IPC App',
        webPreferences: {
            nodeIntegration: true,
            contextIsolation: false,
        }
    });
    ipcMain.handle('two-way-com', async (_e, _arg) => {
        console.log("_arg:", _arg);
        return _arg;
    });
    mainWindow.loadFile('index.html');
};

app.once('ready', () => {
    createWindow();
});

app.once('window-all-closed', () => app.quit());

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="Content-Security-policy" content="default-src 'self'" />
    <meta name="viewport" content="width=device-width, inicial-scale=1.0" />
    <title>IPC App</title>
    <link rel="stylesheet" href="styles.css">
</head>
<body>
    <h2>Input something</h2>
    Send to main: <input type="text" id="send"/>
    <button id="button">Send</button>
    <p id="receive">Receive from main</p>
    <script src="index.js"></script>
</body>
</html>

const {ipcRenderer} = require("electron");
const button = document.getElementById("button");
const receive = document.getElementById("receive");
const send = document.getElementById("send");

button.addEventListener('click', async () => {
    receive.textContent = await ipcRenderer.invoke('two-way-com', send.value); // I mistook `send.value` into `send.textContent` 
});

I will try to combine this code with another program.

Day12~17 of 100DaysOfCode

KoichiArai — Tue, 30 Jul 2024 15:07:41 +0000

I have been struggling with Inter-Process Communication (IPC) for 5 days. (Also dealing with COVID too)
Finally, I have understood how to use methods such as 'ipcRenderer.invoke' and 'ipcMain.handle' a bit.
However, there are still many things to learn.

What I did

Googling about ipcMain.on and event.reply
- Inter Process Communication
- Basic of Inter Process Communication for Electron(v15.0.0) (JPN)

I couldn't understand event.reply even after using chat-GPT. I referred to the following article and used .invoke and .handle instead.

Copying and pasting tutorial code and modifying it
- Main Process and Renderer Process(JPN)

And here is what I made:

// main.js
console.log("Main process working");
console.log("main.js");
"use strict"
const { app, BrowserWindow, ipcMain, dialog} = require('electron');
const path = require('path');
const url = require('url');
let win;

function createWindow() {
    const mainWindow = new BrowserWindow({
        width: 400,
        height: 300,
        title: 'IPC App',
        webPreferences: {
            nodeIntegration: true,
            contextIsolation: false,
        }
    });
    ipcMain.handle('open-dialog', async (_e, _arg) => {
        return "hogehoge"
            // .showOpenDialog(mainWindow, {
            //     properties: ['openFile'],
            // })
            // .then((result) => {
            //     if (result.cancelled) return "Cancelled hoge";
            //     return "hoge";
            // });
    });
    mainWindow.loadFile('index.html');
};


app.once('ready', () => {
    createWindow();
});

app.once('window-all-closed', () => app.quit());

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="Content-Security-policy" content="default-src 'self'" />
    <meta name="viewport" content="width=device-width, inicial-scale=1.0" />
    <title>IPC App</title>
    <link rel="stylesheet" href="styles.css">
</head>
<body>
    <h2>Input something</h2>
    URL: <input id="URL"/>
    <button id="errorBtn">Open</button>
    <p id="text"></p>
    <script src="index.js"></script>
</body>
</html>

const {ipcRenderer} = require("electron");

const Button = document.getElementById("Button");
const text = document.getElementById("text");

errorBtn.addEventListener('click', async () => {
    console.log("index.js");
    text.textContent = await ipcRenderer.invoke('open-dialog');
});

I will try to use the packaging function tomorrow.

Day 11 of 100DaysOfCode

KoichiArai — Tue, 23 Jul 2024 15:36:02 +0000

What I did

Learning IPC Communication Methods

I learned about IPC (Inter-Process Communication) methods today. However, I didn't fully understand how to use these methods. In addition to this, in the tutorial on IPC communication in Electron, I couldn't understand the code block below, especially the portion involving the object and arrow function.

contextBridge.exposeInMainWorld('electronAPI', {
  setTitle: (title) => ipcRenderer.send('set-title', title)
})

I will solve these question tomorrow.

Day9,10 of 100DaysOfCode

KoichiArai — Mon, 22 Jul 2024 03:51:34 +0000

I have set my goals for the 100DaysOfCode challenge. My large goal is to create a tech support system for a friend's company to help prevent it from going bankrupt. My small goal is to understand how to make applications with Electron and React Native. Therefore, I will focus on learning how to make apps with Electron and React Native for the rest of the 100 days.

What I did

Built Electron environment

Watch videos
- Electron JS: Build Amazing Desktop Apps
- Making Desktop App using Electron
Read documents
- What is Electron?

What I learned

How to make a project on Electron
Understanding the main process and renderer process in Electron
How to make a package

Day8 of 100DaysOfHTML

KoichiArai — Wed, 17 Jul 2024 13:33:30 +0000

Go with someone absolutely when you are going to enter mountain.

What I did

Begin to watch new video Learn HTML

What I learned

What HTML means
↓↓↓

<html></html>
<h1></h1>
<h2></h2>

It's better to do something than nothing I hope...

Day7 of 100DaysOfJavaScript

KoichiArai — Mon, 15 Jul 2024 15:19:38 +0000

I have accomplished watching the video.

What I did

Continue watching video Learn JavaScript

What I learned

Math.random and Math.floor
Concise format of defining if statement

if (hoge === hogehoge) {
  return true;
} else {
  return false;
}
// concise format below
return hoge === hogehoge ? true : false;

object.freeze()
Arrow function
.filter()
.map()
Rest operator
Spread operator
Destructring
Template literal

I will start to learn HTML and CSS tomorrow.

Day6 of 100DaysOfJavaScript

KoichiArai — Mon, 15 Jul 2024 00:58:00 +0000

Lately there are many sudden rain...

What I did

Continue watching video. Learn JavaScript

What I learned

Nested arrays
for and while loop

for (var i = 0; i < 10; i += 2) {
  console.log(i); // 2, 4, 6, 8
}
for (var i = 10; i > 0; i -= 2 ) {
  console.log(i); // 8, 6, 4, 2
}

nested for loop I have frequently used this when I conducted graduation study.
Random (Math.random, Math.floor)