Hosting a Twitter bot for free on Heroku

#javascript #showdev #node #twitter

Cover Photo by Kaboompics.com from Pexels

Originally posted on my learning journal: https://journal.fullsnacktester.com/learning/2021/01/30/twitter-bot.html

!! Please note, Heroku are removing their free dynos, so you can no longer run a Twitter bot on Heroku for free. See https://blog.heroku.com/next-chapter

Following a tutorial

I started writing the @TesterOfTheDay Twitter bot by following a tutorial. I am not ashamed to admit it, I wanted to make a Twitter bot, not learn how to make a Twitter bot. So I used the final state of the tutorial as my starting point.

You can find the tutorial I followed here:

How to create a Twitter Bot using Node.js and Heroku

I have no intention of repeating anything from that tutorial, if you want to follow it, go read it. Of course I changed it enough so that it searched for #TesterOfTheDay.

If you want to start from my final state, go right ahead and grab the code from GitHub:

testeroftheday-twitterbot

Using dotenv to protect credentials

The tutorial used a config.js method for providing the app with Twitter API keys, with a vague note not to leak your credentials. As I wasn't familiar with how to use a config.js in a secure why while hosting my repo openly on GitHub, I went with what I know and used dotenv.

For local development I have a .env file that I do not check into GitHub. The way dotenv works, if a .env file is present it loads the content into environment variables, accessible in NODE via process.env.

require('dotenv').config();
const twit = require('twit');


const T = new twit({
    consumer_key: process.env.CONSUMER_KEY,
    consumer_secret: process.env.CONSUMER_SECRET,
    access_token: process.env.ACCESS_TOKEN,
    access_token_secret: process.env.ACCESS_TOKEN_SECRET
})

I then configured the environment variables (my Twitter credentials) in Heroku. For this I followed the Heroku documentation: Configuration and Config Vars

Switching to 10 minute schedule for huge savings

The problem

By default Sumedh Patkar Twitter Bot uses setInterval to trigger once every minute. This gives a very fast response, and is infrequent enough to comply with the Twitter API rules, as far as I can tell at least. This method has one big consequence, it was burning thorough my free dynos, aka credit on Heroku.

The solution

I changed from using an infinite loop to a one off execution that could be triggered by the Heroku Scheduler.

Note to install the free Heroku Scheduler plugin, I did need to add a Credit Card to my account. This had an added benefit of doubling my free dynos.

Now the scheduler in the free plugin only allows a trigger once every 10 minutes, but for the Tester of The Day bot this was still plenty. Realistically there are only a small handful of Tweets on that hashtag every day and no where near enough to justify a 1 minute repetition.

You can see for yourself the minor code change on GitHub in this commit:

https://github.com/dowenb/testeroftheday-twitterbot/commit/ea095b8572f29555e548c2b61afae386406fea3f

It also required following the docs and setting up the Scheduler plugin. The job simply runs node bot.js every 10 minutes and was trivial to setup once I had added my credit card.

Make sure you also turn off the web and worker Dyno formation. You don't need that running, because execution now comes only from the scheduler.

The savings $$$

On a free Heroku account you get 550 "dyno hours", raising to 1000 when you register a credit card.

In a month with 31 days, running the Twitter bot 24 hours a day in a loop would consume 24 * 31 hours, that is 744! Even when adding my credit card, this wouldn't of left me with much credit to host anything else for free.

With the change to running one every 10 minutes, in January my Twitter bot consumed only 51 hours. What is more, that includes over one full day wroth of hours a the start of the month before I made the change!

So I've one from using up all my credit, to gently sipping away at it. I expect in February I'll see the full benefit with even further reduction in usage.

In conclusion

If you don't need a process to respond to web requests, and you don't need it to be running 24/7 consider using the Heroku Scheduler. It could allow you to run for free what might otherwise cost your more, or reduce your usage if your are already on a paid tier.

While I found this change valuable on Heroku with my Twitter bot, the principle is potentially transferable to other situations. Let me know how you get on!

Tester of The Day

While I am here, let me tell you, Tester of The Day is a daily award that celebrates great people who contribute to the Software Testing community. Winners are induced into the Tester of The Day: Hall of Fame. Check it out!