Introduction
The internet is the most valuable source of information in today’s society, yet it can be time consuming to search webpages for the right information. However, Python provides powerful ways to scrape these webpages to get what you need. Additionally, we can use WayScript to create a program that uses these Python web scrapers and automatically transfers the results to a Google Sheet, keeping your information neatly organized in one place. Keep reading to learn how to do this!
Building our Script
The first thing that is added to the program tree is a Time Trigger module. This module allows the script to automatically run at set time intervals. To set the time, go to the left toolbar and select how often the script should run, the start time, and the specific time zone. Now that is set, a Python module is added so the web scraper can be enabled. We’ll edit the code in this module by clicking on “edit code” to the left.
This script requires the user to make a request to a website, so the request library must be imported through WayScript. This is a fairly easy process. Simply type “import requests” in the first line of code to gain access. The next lines of code tell Python to scrape whatever website is preferred.
When the code is run it gives a status code for the website to determine if the site is correct and running. Now, this Python variable needs to be converted to a WayScript variable in order to successfully transpose the results to a Google Sheet.
To convert the Python variable to a WayScript variable, a simple line of code is all that’s needed:
variables[‘status_code’] = response.status_code
where “response.status_code” is the Python value created in the previous step. When this code is run the new variable appears in the bottom left of the toolbar. The next module added to the tree is the Date Time module. This creates another variable that’s used in the next step.
It’s now time to add the created variables to a Google Sheet. Using the Google Sheets module, select the appropriate file and worksheet to write the data to in the left toolbar. Remember the created variables from the previous step? They’ll now be added using the dropdown menu in the left toolbar.
Click the play button on the Time Trigger module to see the script run in real time to the left and on the Google Sheet. To run the script on the set time schedule simply activate the Time Trigger module slider to the on position. The script will now automatically run for the set intervals.
There’s no need to worry about spending time scouring the Internet for information anymore. While other programs offer the ability to web scrape, WayScript provides the means to effortlessly create scripts that can do this while utilizing other programs simultaneously. This gives users a wider range of usability for their everyday business applications.
Conclusion
Questions about this script or anything else? Join our discord. We're always around to help. If you want to work the full script template, just find it here.
Top comments (0)