Get started within 30min
It’s always interesting to get an update on the different Cryptocurrency pricings. Sadly enough there are not that many services out there that do that out of the box for us, so let’s create one!
Prerequisites
- Scraper.AI account
- AWS Account
- IFTTT account
Flow
Once the prerequisites are met, we can now set-up a flow that will scrape the cryptocurrency pricings from https://coinmarketcap.com/ every hour and send a Tweet once these got received. To do this, we need to create a flow that looks like this:
Scraper.AI -> Data Processor -> Twitter
The easiest way to do this is to have a Serverless function that gets executed on the Scraper.AI Webhook, which will manipulate the data and send this to Twitter. To send to Twitter, we utilize the IFTTT service, which has a connector to send a Tweet. In components we will thus get something like this:
Scraper.AI -> AWS Lambda -> IFTTT -> Twitter
Setup
Setting up AWS Lambda
The first thing we want to do is to set-up our AWS Lambda function. Here we want to configure a function that can be called through an HTTP endpoint, so we have to add a Trigger with the API Gateway to it.
Once we then have our boilerplate and trigger configured, we can add a package.json and index.js
Note: you will have to create this locally, install the NPM modules with npm install, archive it into a .zip file and upload this to the Lambda portal.
This was the hardest part! The rest is just click-click-click 😉 so let’s continue!
Setting up IFTTT
IFTTT (If This, Then That) is an easy service to utilize. We can create an applet here that takes the URL called above in our AWS function (which we will get at the end here) and send it to Twitter.
Let’s configure IFTTT
Now Twitter has been connected, the last thing we need to do is to get the Link for our Webhook.
Note: this URL you will have to enter in the AWS Lambda Function!!!
Setting Up Scraper.AI
Last but not least, we need to monitor https://coinmarketcap.com/ to changes. Luckily we can utilize https://Scraper.AI here!
Navigate to https://coinmarketcap.com/ and select the properties name, price and volume
Then continue until the Website is scraped and displayed on Scraper.AI
Note: Configure scheduling to happen hourly! This ensures that you post the newest tweets every time.
Finally, go to Notifications and enter the AWS Lambda endpoint you copied earlier into the “Webhook” field.
Congratulations! Tweets should now start appearing into your Twitter account.
Note: to test this easily, you can “Scrape Manually” on the dashboard!
An example of this can be seen below:
Top comments (0)