Hey Bilal, nice overview. I noted something odd that I wanted to ask you about. The module time is in the standard library as time and so is time.sleep(). I'm not sure wat you're installing through pip. I checked on pypi but it's not clear.
Hello rhymes, thank you for noticing that. I double checked, and indeed the time module is part of the standard library. docs.python.org/3/library/time.html
I have edited the post. Your feedback is appreciated.
Thanks a lot, Bilal for sharing this. Curious to know if this solution is OS dependent.
Hello Suhail, you're very welcome.
I am on my Mac right now, so I had the chance to do a quick test.
All I had to do is download Chrome Webdriver for MacOS from the link provided in the post, installed selenium with pip. Of course I had the change the path to the webdriver, I ran the python script and it worked with no additional tweaking.
Another tip, if you'd like to run the job with Chrome headless:
# imports here
# links = [array of links]
options = webdriver.ChromeOptions()
with webdriver.Chrome('/Users/userName/chromedriver', chrome_options=options) as driver:
# code here
I tested the headless on MacOS, I am trying to test this on Ubuntu on a remote server. I'll update the post if I succeed.
Nice work Bilal, Thanks a lot for the effort.
Suhail, I have updated my post if you're interested in Headless Chrome.
Thanks a ton Bilal :)
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.