Tutorial: Running Selenium in the cloud on a daily cron schedule

Start your Monday off with a super short tutorial using Selenium on WayScript. Questions about WayScript? Check out our FAQ.

Introduction

Selenium is a library for automating web browser tasks. With WayScript, you can run selenium code on the cloud on an automated schedule without needing to set up a server or any other infrastructure.

Setting up your Cron Scheduler

Under triggers, select the Time Trigger.

Next, set the cadence and time that you want your script to run at (ex. every day at 9am, or once a week on Thursdays, etc.).

Setting up your Python Script

Drag the Python Module into your WayScript flow

Setup your Selenium Headless Browser

from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
​
browser = webdriver.Chrome(options = options)
browser.set_page_load_timeout(10)
​
browser.get('https://wayscript.com/')
​
ps = browser.page_source
print(ps)
​
browser.close()

All done! Now your selenium code will run

You can also output data from Python to use in other modules such as sending emails, slack messages, etc.

Comments

[Nicolas Carmont]

I get this error, any ideas on fixing it?

Traceback (most recent call last): File "python_step", line 10, in browser.get('wayscript.com/') File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 296, in get self.execute(Command.GET, {'url': url}) File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 284, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 205, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash from unknown error: cannot determine loading status from tab crashed (Session info: headless chrome=84.0.4147.89)