Scrape Web App Data & Add it to Google Sheets with WayScript

Introduction

Scraping data from Twitter is very useful when you want to make sentiment analysis or understand a phenomenon taking place online. However it requires a good level of coding and a better comprehension of changing websites structures. Using Wayscript, we can bypass these problems, and seamlessly integrates Twitter and Google Sheets. Today we’ll create a script to automatically scrape Twitter for social mentions add to Google Sheets.

Prerequisites

No prerequisites but some content you might find useful:

• Working Google Sheets
• Workig with Twitter

Building our script

Our workflow will look like this:

• Add a time trigger to periodically run the Twitter Scraper module
• Use the list returned by the module and loop through it to get each item
• Use important data about each item and add it to sheet files

You can work with any trigger you want like a text message trigger or Gmail trigger, but here we’ll use a time Trigger.

The Twitter Module

The next step is to scrap Twitter according to a special mention or search item. For this purpose, we’ll use Twitter Module provided by Wayscript to get the latest 50 tweets about cryptocurrency.

Set the mode to Search for word or #hashtag. In advanced input parts, you can select options such as ignoring retweets, replies and choose the result type (mixed or popular).

We've selected the Tweet outputs to get a list. It will be more easier to work with. You'll notice some changes in variables panel.

Add to Google Sheet

The Twitter module will return a list of 50 tweets, each item of the list containing a lot of information. Here, we will only use the username, the name, the content, and the date.

But first, we must add a loop step.

Then in the loop add the Google Sheet module and set it to Write Mode.

Now let’s add the variables. Understand that here each variable is a column.

Final Step

Now turn on the time trigger. The automation of this script is handled by Wayscript. Then, the google sheet will be updated every day at 11.00 am.

Questions or concerns?

Questions about this script or anything else? Join our discord. We're always around to help. If you want to work the full script template, just find it here.