Scraping data from Twitter is very useful when you want to make sentiment analysis or understand a phenomenon taking place online. However it requires a good level of coding and a better comprehension of changing websites structures. Using Wayscript, we can bypass these problems, and seamlessly integrates Twitter and Google Sheets. Today we’ll create a script to automatically scrape Twitter for social mentions add to Google Sheets.
No prerequisites but some content you might find useful:
Our workflow will look like this:
- Add a time trigger to periodically run the Twitter Scraper module
- Use the list returned by the module and loop through it to get each item
- Use important data about each item and add it to sheet files
You can work with any trigger you want like a text message trigger or Gmail trigger, but here we’ll use a time Trigger.
The next step is to scrap Twitter according to a special mention or search item. For this purpose, we’ll use Twitter Module provided by Wayscript to get the latest 50 tweets about cryptocurrency.
Set the mode to Search for word or #hashtag. In advanced input parts, you can select options such as ignoring retweets, replies and choose the result type (mixed or popular).
We've selected the Tweet outputs to get a list. It will be more easier to work with. You'll notice some changes in variables panel.
The Twitter module will return a list of 50 tweets, each item of the list containing a lot of information. Here, we will only use the username, the name, the content, and the date.
But first, we must add a loop step.
Then in the loop add the Google Sheet module and set it to Write Mode.
Now let’s add the variables. Understand that here each variable is a column.
Now turn on the time trigger. The automation of this script is handled by Wayscript. Then, the google sheet will be updated every day at 11.00 am.