DEV Community

Automated Dapps Scrapping with Selenium and Metamask

Leonardo Teixeira Menezes on July 28, 2021

One of the newest trends in web development is the rise of decentralized applications, also known as Dapps. These applications are built leveraging...
Collapse
 
fedeblengio profile image
Federico Blengio

Thanks bro, it has helped me a lot, but I have a problem for days I am trying to solve it but I can not.

any ideas?

Collapse
 
ltmenezes profile image
Leonardo Teixeira Menezes

Hey Federico, good to know that this article is helping you, thanks for the feedback!
This kinds of issues might happen because the behaviour of the extension might differ depending on the metamask extension version.
In these cases the best approach is to use some python debugger (such as IPDB) to stop the code for you to find exactly what is different in your version, most times you only need to do some minor changes.
Just followed you, feel free to drop a DM if you still have the issue!

Collapse
 
fedeblengio profile image
Federico Blengio

here is my problem
prnt.sc/1oab8yn

Collapse
 
baotran810 profile image
baotran810 • Edited

Hi Federico,
I also got the issue "NoSuchElementException" because it couldn't locate the "Get started" button. While debugging, I found that it focused on the first page when I ran the script, so I switched to the first tab to close then it only focused on the MetaMask page then the "Get started" button was able to locate. Here is the script I used, I hope it would help you.

await driver.getAllWindowHandles().then(function (handles) {
driver.switchTo().window(handles[1]);
driver.close()
driver.switchTo().window(handles[0]);
});

Thread Thread
 
fedeblengio profile image
Federico Blengio

Thank you

Thread Thread
 
fedeblengio profile image
Federico Blengio

I have a question to ask about handles, you can send a dm?

Thread Thread
 
escaraptor09 profile image
Marc Ferri

How did you finally solved it?????

Collapse
 
escaraptor09 profile image
Marc Ferri

I got the same problem

Collapse
 
p12 profile image
P12522 Brasil

Awesome!!!

Collapse
 
techiesworld01 profile image
TechiesWorld01

Hi Leonardo,
I am facing issue while installation of selenium through chrome driver.

ImportError: cannot import name 'webdriver' from partially initialized module 'selenium' (most likely due to a circular import) (C:\Users\Admin\Desktop\selenium.py)

Collapse
 
bacolen profile image
Raphael Bachofen • Edited

So helpful! Finally someone posted this...

Collapse
 
andrew166 profile image
Andrew

what is use of this ?

Collapse
 
berengueradrian profile image
berengueradrian

Thanks a lot for the code. I am having some issues with this approach, I write this comment to see if someone has had a similar issue or can help me with mine.
See this: stackoverflow.com/questions/762522...
And this: dev.to/berengueradrian/runtimecall...

I need to solve this for as much on thursday, it would be really helpfull if someone has the solution, thanks a lot in advance!

Collapse
 
nmolanog profile image
Nicolas Molano Gonzalez

I am lost with this:
"Click 'Pack extension' and enter the local path to the Metamask extension This will generate a .crx "

When I click Pack extension there is a window which ask for Extension root directory and private key (optional)

I am confused on how this should be done. I need to know where is the local path in my pc where metamask is? if this is the case how can I find this location?

Collapse
 
nmolanog profile image
Nicolas Molano Gonzalez

I was able to find it but I am getting a .pem file instead of an .crx

Collapse
 
alirio_miguel profile image
Alirio Aranguren

I'm having an issue using Slenium and Java... Whenever I click on the staking button it says that the contract needs to be approved, although it alredy has... I think that it has something to do with the extension but I'm not 100% sure
Image description

Collapse
 
alirio_miguel profile image
Alirio Aranguren

this is what should be displaying
Image description

Collapse
 
cryptotingle profile image
CryptoTingle - 💢 #RouteArmy

Hey guys! If you have problems with the above try this. Just remember to enter your own extension path for metamask (C://...whatever) and your own 12-word seed phrase. Thanks OP! Enjoy!

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

import time

EXTENSION_PATH = 'C:\......INSERT YOUR PATH HERE'

opt = Options()
opt.add_extension(EXTENSION_PATH)

driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=opt)

driver.switch_to.window(driver.window_handles[0])
original_window = driver.current_window_handle

Get Started

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='app-content']/div/div[2]/div/div/div/button"))).click()

Import Wallet

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='app-content']/div/div[2]/div/div/div[2]/div/div[2]/div[1]/button"))).click()

Donate Data. No thanks!

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='app-content']/div/div[2]/div/div/div/div[5]/div[1]/footer/button[1]"))).click()

time.sleep(1)

Seed phrase

driver.find_element(By.ID, "import-srp_srp-word-0").send_keys("FIRST WORD")
driver.find_element(By.ID, "import-srp
srp-word-1").send_keys("SECOND WORD")
driver.find_element(By.ID, "import-srp
srp-word-2").send_keys("THIRD WORD")
driver.find_element(By.ID, "import-srp
srp-word-3").send_keys("FOURTH WORD")
driver.find_element(By.ID, "import-srp
srp-word-4").send_keys("FIFTH WORD")
driver.find_element(By.ID, "import-srp
srp-word-5").send_keys("SIXTH WORD")
driver.find_element(By.ID, "import-srp
srp-word-6").send_keys("SEVENTH WORD")
driver.find_element(By.ID, "import-srp
srp-word-7").send_keys("EIGHTH WORD")
driver.find_element(By.ID, "import-srp
srp-word-8").send_keys("NINTH WORD")
driver.find_element(By.ID, "import-srp
srp-word-9").send_keys("TENTH WORD")
driver.find_element(By.ID, "import-srp
srp-word-10").send_keys("ELEVENTH WORD")
driver.find_element(By.ID, "import-srp
_srp-word-11").send_keys("TWELFTH WORD")

Passwords

driver.find_element(By.ID, "password").send_keys("password")
driver.find_element(By.ID, "confirm-password").send_keys("password")

Agree to T&Cs & Submit

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='create-new-vault__terms-checkbox']"))).click()

Import Button

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='app-content']/div/div[2]/div/div/div[2]/form/button"))).click()

Congratulations page

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='app-content']/div/div[2]/div/div/button"))).click()

Intro pop-up box

WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='popover-content']/div/div/section/div[2]"))).click()

Collapse
 
f041 profile image
Gabriele Giordano

I get, when trying to input a password in metamask password box:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"#password"}

even though I tried numerous other paths:

Tried by xpath:

//*[@id="password"]

//*[@id="app-content"]/div/div[3]/div/div/form/div/div

('//input')

Tried by ID:

Tried by selector

#password

Collapse
 
yournicespice profile image
YourNiceSpice • Edited

Hey Leonardo,I had a problem when I went to the dapp he offered a connection with metamask,
i called
driver.get ('chrome-extension: //nkbihfbeogaeaoehlefnkodbefgpgknn/notification.html#connect/')
i saw connection window
But after that chrome webdriver breaks

Collapse
 
hehe2046 profile image
yoyo

What are your versions of Python and Selenium in the cain?

Collapse
 
agustinustheo profile image
Agustinus Theodorus

I needed this. Thank you very much!

Collapse
 
andrew166 profile image
Andrew

Hi,
can you tell me what is profit of this ? anyone explain plz?

Collapse
 
aerryasmani profile image
Aerry

Thank you for the articles. Is there anyway that this method can be applied in Katalon Studio as well? Cause I'm new in testing and we're using katalon as a testing tools.